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THE WORK STATEMENT 

Thi* proposal consists of four wjor components: 

■ The Artificial Intelligence Laboratory proposes to work on 
issues central to the large data base problem, the 
Intelligent terminal problem, and the natural language 
engineering p rob lea with a broad h integrated research 
progran- There are several imediate tasks: 

To produce a natural language interface for accessing 
intelligent terminal and large data bas.fi tools. 

To produce a natural language generating progran to act with 
the understanding program. 

To develop Wlnsky's frane systen. into a forn suitable for 
dealing with large bottles of knowledge r 

To create resoursa allocation program suitable for 1 
variety of scheduling tasks. 

To create models of people and organizations auch that 
-programs can understand their needs - 

■ The Artificial Intelligence Laboratory proposes to worfc on 

applications of computer vision with the enpheili shifting 
from tool building toward solving real prcblens. Several 
new tasks are to be undertaken; 



To create in inspection program for dealing with 

sophisticated jet engine testings. 

To create representations for terrain such that vehicles can 
discover their own orientation using it. 

To develop tracking prog reus that use knowledge About the 
objects tracked beyond simple velocity and acceleration 
prediction. 

The Artificial Intelligence Laboratory proposes to continue 

a basic studies program Aimed at dealing with deep, central 
Issues. The issues are to be exposed by work in computer 
vision, common sense reasoning! and expert problem solving. 
The ubiquitous representation problem is involved in ell 
three areas. The basic goals include the following: 

To create solid symbolic representations from images - 

Ta create programs which can do comno-n sense reasoning about 
the simple physical world- 

To create expert problem salvers that expose the general 

Issues pervasive in the creation Of all engineered systems. 

The Artificial Intelligence Laboratory proposes to complete 
development of Law cost hardware intended to serve single 
users with an outstanding LISP computing system, drastically 
Superior to any available today both in terns of power and 
cost, 
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THE PROPOSAL 

This proposal specifies both our long, terra direction and our specific goal* 
as planned for the next two years. Ve relate these goals to past progress, 
iitaediate practical needs, end long tern scientific objectives. Since nany 
of the scientific objectives cited are deep end demanding, readers should 
understand that considerably nore than two years will be required for full 
solution. 

THE FOUNDATIONS 

From the beginning, our purpose has been to make machines oore Intelligent, 
partly to make then more useful and partly to learn about ibe nature of 
intelligence.. Certainly the need to make machines smarter is clear. It Is 
also clear that the ideas or Artificial Intelligence are the principle 
scientific path toward Baking machine* smarter, Warn aoora v 1 1 1 it ftapptn ens' 
Ar.u nracfr miEE ft cost? 

What nakos prediction difficult, of course, is the non-linear 
relationship be.twe.an the level of technical devoid pliant and the payoff. For 
any combination of Artificial Intelligence paired with a need, there is scut 
thros-hold on costs and capabilities such that on one side., little happens, 
and on the other, resolution r The pocMt calculator phenomenon, is one 
example of this that is known to everyone. The pocket computer Is around 
the corner. He believe that there are many areas in which Artificial 
Intelligence will produce it* own revolutions resembling that of the pocket 
calculator in their swiftness . Two particularly promising areas are these: 

■ The combination of large data base handling, intelligent 
Interfacing of software tools and people, and natural 
language interface engineering. 



Tills revolution wilt be based on i qualitative advance in which programs 

understand, to a significant degree, the information they are handling; the 
current technology merely applies Banipulativ* rituals that d* not use much 
of the knowldege embedded in the data bases being manipulated. 

■ The application or the- ideas of artificial intelligence to 

computer vision and manipulation for productivity 
tec hn ol guy i vehicle guidance, and inage understanding. 

Again, the breakthrough will defend en the use of representations of the 
■leanings, uses, and special features of the objects to be sensed and 
■inipulated. 

Those are only what we view as immediate areas of naxlnum opportunity. 
Many others are suggested by the following sample: 

SOHE ARIAS OF APPLIWTIOM 



Hanufacturlng 

Kining 

Firming 



Repair And fti in tenants 



Assembling Electronic Systems 

Assembling Small Herhanical Devices 

Coal Mining In Dangerous Nines 

Undersea Recovery Of Jlanganese Modules 

Selective Harvesting With ffixed Crops 

Pest Control 

Pruning 

Vehicle Debugging 

Floor Care 



Purposeful Wnnitaring 



Autoaatic Programming 



ManagEEiejH Assistance 



Logistics 



Hedicine 



Education 
Fir* Fighting 
Theft Prevention 



Intelligent Autopilots 

Shipboard Functions 

H&nagement Reports 

Inventory Control 

Production Scheduling 

Quality Control 

Very Large Data Bases 

Intelligent PERT Networks 

Scheduling People and Groups 

Question Referral 

Kews SuBimariiina 

Docuiient Draft Polishing 

Routing 

Intelligent Substitution 

Equipment Sharing 

Diagnosis 

Intensive Care Monitoring 

Treatment iHnagemant 

Radiation Therapy Setup 

Intelligent CA1 Systems 

Information Retrieval 

Fearless Firemen 

Apparatus Allocation 

Tracker* 



Mow can such oh j actives ha reach ad? Certainly for smart machines tn wait a an 
Impact in areas like those mentioned, the snart na chin as must exhibit a. 

variety of talents associated 'Kith intelligence. Computer vision, English 

understanding, learning and debugging, and expert problem solving all must 
ba undent odd. It is natural then that considerable effort has gone into 



undoratandino these things. Kappily *# get back to them both hy way of our 
desire to lay the foundation far applications and by way or our hope to 
understand intelligence for its awn s*ks, 

GENERAL OBJECTIVE: TO nME MACHINES SflARTiR 



TO MAKE HACKINES MORE USEFUL 



Tff UNDERSTAND- INTELLIGENCE 



Areas Or Application 



System! To Be Understood 



Intelligent Terminal* 

Very Largo Data Base* 

Remote Guidance 

Photo Interpretation 

Logistics 

Purposeful Monitoring 

Repair And Maintenance 

Cleaning 

Manufacturing and Mining 

Medicine 

Education 



image And Scene Analysis 
Speech And Text Understanding 

Learn Ln-g and Debugging 
Expert Problen Solving 
Coiunon Sense Reasoning 
Goal Understanding 



A£ the diagram suggests, it has been sensible during the gestation years to 
be organized around tbeaes lilts coapatcr vision, English undtatending. 



learning and debuUJiliiJ, add yxp-flrt problem Solving. It is through these 
particular thrusts that >-e- hiive Leurned j ^rtat ■Iflfll about what are now 
regarded as Central theoretical Issues: 

SW1E CENTRAL THEORETICAL ISSUES 



Knowledge Representation 
Starch And Control 
Programming Methodology 
Grouping 



Constraint Exploitation 
Classification And Deduction 

Goal Directed Versus BottOQ Up 
Serial Versus Parallel 



THE FUTURE 



The Structure 1 Of Th* Li torn 1 o Ftf is EvoivlitQ With The Science 



Thei Artificial Intelligence Laboratory is widely recogbiied is. one of th* 
world's principal sources of ideas, experimental prototypes > and feasibility 
demonstrations on the frontier of COBputer science- It would not be 
appropriate for the laboratory to concentrate on producing operational 
systenfi in aroas llfce that of very large data bases, but it dOQS S«&D 
appropriate to begin thinning in terns of what AI can lead to in the near 
term and to hog in producing the basic research results lying Just in front 
Of prototype production. To this end, we now sea ourselves organized Into a 
n** structure that draws not only frop the desire to understand varloui 
dimensions or Intelligence but also from e more direct interest in 
applications challenges on one aide and an interest In the oaorging central 
Utuea on the other. Manifestly the boundaries are to be sort, not hard, 
and as has been common in the past, much flow among the particular subgroups 
is expected. 



Applications Orisntei Basic Studies 

•i 

Because natural language research has aide muth progress along with parallel 
advances in our theories Of knowledge, a considerable portion Of pur 
™" ur " s I*WW be assigned ta a unified progra» Oil very large data bases, 
intelligent terminals, and natural Language in terrace engineering. Part or 
our efforts for the reminder or 1575 -ill be directed at bringing this area 
Lip to 3Dned. 

Application Oriented Vision Ani Hvni pulnUtut 

Vb have spent considerable time and effort in developing a. laboratory 
facility that is at once powerful, easy to copy, and relatively inexpensive. 
It has been developed using electronic circuit board assembly a; the test 
problem. It is now tine to redirect the use of this rtciUty to other 
pressing problems by selecting the best of many alternatives for allocation 
Of this impart ant resource. Fully automatic assembly of small mechanical 
devices seen* appropriate. Part of the liege understanding problem also 
setae to be promising. 

Itmti Oriental Basic Studies 

The study of basic issues with domains selected to facilitate understanding 
remains crucial to our program. With.in. this collection of topics we sea 
continued work on the representation pf knowledge, perhaps our mast pressing 
problem area, as well as on expert problem solving, caramon sense physical 
reasoning, problem salving hy analogy learning, and constraint driven 
problem solving of the sort exposed by studies in computer vision. This 
last mentioned work on computer vision is believed to be critical for long 
range progress on Visual interpretation problems that require nearly all of 



a large array of image points to be processed, 

SifstEJxs And Sjwien* Concepts 

Progre** in Artificial Intelligence is facilitated by adequate equipment. 
Indeed. many pf its great accompli Shm-nts have involved, program that are 
too large and to complex to be produced without the rrontiar tine sharing 
concepts wo have developed. So* time sharing; Is well established, but 
newer, even nore revolutionary opportunities have eoergedL It there tore 
seems wise to continue our systems concepts leadership, both because of our 
own needs in Artificial Intelligence and because of the profound impact wo 
believe these computer Ideas will soon have on the computer con tuning 
Coamu-nity. 

RECENT PEOCHE^ 

Thi-5 introduction closes With a summary Of recent progress organized around 
the topics which correspond tn our past foci. These are: 

Representing Knowledge and Expert Problem Solving 

Understanding Natural Language 

Understanding Vis- ion 

Productivity Technology 

Learning and Debugging 

Systems and Architecture 

and New Programs 



fttprewntlflt RrtfHBlEdgE ArJ Expert profiler* Salving 

Sussman and his Students have nade progress in the direction of 
understanding debugging and the expert problem solving process in the 
particular domain nf electronic circuitry. Sussman and Stallmaii, for 
example, completed a program and a paper in Harch OT 1975, "Heuristic 
Techniques in Computer Aided Circuit Analysis." The program understands 
circuits well enough to determine proper operating voltages and currants by 
cunmon -sense reasoning, rather than fcy solving network equations. It can t 
now handle the rather eoopllcated circuits found In IC chips, Specifically, 
the circuitry of a UA741 05 odd has been successfully analyzed, Experts in 
the electronics field have shown considerable interest in this now, 
knowledge-based, common sense approach. Sussman believes that the 
principles will transfer smoothly to other donalns and become a general 
theory of debugging applicable to all engineered systems. 

Fahlaan and Grossaan have done extensive work on a new representational 

scheme which we believe may have considerable consequences with respect to 

future machine architectures, Fahlaan and Grossman argue for a network of 

nodes and relationships from which knowledge is retrieved by parallel 

searches along A-KlND-OF and other links. Their thoughts have been 

geminating, over the past year, but the work was too preliminary to allow 

commitment to writing until recently. Now both Fahlman and Grossman have 

completed papers which outline theories, propose experiments, and define, in 

the case of Grossman, specific application possibilities in the very large 

data base area. New hardware progress in the integrated circuit industry 

provides soma hope that machines oriented toward such searches nay become 

practical p given that simulation on serial nachinos proves the ideas sound. 

sandewall has concentrated on the data base problen as well h but with 

with a view toward dealing with sone immediate problems of data base 

organization and self description, He has worked out a block structured 



scbena capable of dandling hofd procedures and data, designed to smooth the 
interface between programs on one side and a collection Of data bases on the 
other. It is expected that his evolving theory will particularly influence 
OUr proposed work un very large data bases. Ssodewall has finished a. 
prototype system of programs that illustrate his Ideas, 

Unde rs toird i ji g Natural Lannuaac 

Pratt and hi£ Students hiivu concentrated On the problem of dealing with 
efficiency issues In parting natural language text. This effort tikes us in 
the comfortable direction or preparing for the time when real application of 
natural language systems Kill make a theoretical understanding of the 
efficiency issue of great practical importance. Pratt's paper of January, 
1975, presents in optlnality proof demonstrating that LTWGDL's parsing 
algorithm is optimal among those of its class in the sense that all the 
phrases that it builds up can be Used. 

Meanwhile ftarcus has completed the programming of a now parsing system 
based on the wait-and-see philosophy fit? handling ambiguity. Instead of 
picking one option at all decision points «lt?i the risk of being stucfc with 
costly retreat, Karens' system carries all options forward, making a 
decision only when the decision can he made with certainty. The design of 
this parser was finished by the end of 1374 1 implementation toot place 
during 1975 and is just now complete. The next task is to awgnent thfi snail 
grammar Marcus has used in debugging his parser with a much mora complete 
gracmar, one larga enough to ha a good interface to various Intelligent 
terminal or large data base modules. 

Progress has also baefi nada on the relatively neglected linguistic 
problem of generating lenguage. This involves, among other things, a need 
to know what the listener already knows. McDonald has completed an R.&. 
Thesis Studying the p rob lei, Thar 8 is no program, embodiment of the ideas it 



puts forth, taut creating Hid aJcperlnan tltlfl with such a. program seems 
sensible and is planned. 

Understanding Vision 

Gene Freuder has developad a system for visual recognition, with 
im plications for the general problem of control. The system is intended to 
provide the flexibility required CO deal with realistic visual scenes;, he 
is looking at htEmerSj not stylised models, but everyday, toolbox hammers, 
under natural conditions (e.g. lighting, background, orientation* 
occlusion). Hi is studying directly the interaction of knowledge and 
control re-quired in an integrated pass at the entire ta-sk, rather than 
looking at the attendant problems of image processing In detail- Particular 
results, as they are acquired, combine With genertl visual knowledge to 
"suggest" and "advise" further processing r This process, called "active 
knowledge," extends and implenents the principles of "heterarchy" and 
"domain directed processing' pioneered in this Laboratory. 

Since description has long been regarded, as the key to success with any 
difficult problem, in artificial intelligence* Hollerbach 1 s work on 
describing curved objects is mother important step forward. He has 
developed an approach towards shape description based on prototype 
modification and generalized cylinders^ a notion invented by Blnford at 
Stanford- His programs describe and identify pottery from vase outlines 
well enough to be remarkably consistent with the descriptions and 
identifications given by arch eo legists, ihe resulting descriptions seem. 
very natural and suitable for dealing with many sorts of objects besides 
pottery. The emphasis throughout has been to develop useful, qualitative 
descriptions which bring out the significant feitures and subordintte lesser 
ones, 

Harr and his students Continue to look at vision from another point of 



view, thit of concentration on thoroughly understanding the problem of 
translating image arrays into symbolic descriptions without the intervention 
of high level knowledge. Their thesis is tnat real image understanding 
cannot be done without very solid programs working at this level. This 
basic wort seen* directly dpi ths critical path leading toward real image 
understanding, be it of ordinary scenes or ERTS type data- As of January, 
1975 N the work had concentrated on discovering what features or one- 
dinenslonal intensity profiles could reliably provide inforwtion about edge 
types and such modifiers as shar&nsss of edge focus. Since then, the edge 
and edge modifier wort has been extended to two dimensions With satisfactory 
results- Ths most important work, however, has been on the problem of 
grouping the very lowest level symbolic descriptions of edges together Into 
larger symbolic aggregates and the subsequent realization that a new theory 
of texture will result from this work. Tlarr has invented about four new 
grouping algorithms and prototype programs have bean written for them- Some 
experiments with these prototype programs have been encouraging, but for the 
experimentation needed, the programs will have to be rewritten for the sake 
of efficiency, working with image arrays is tine consuming and requires 
mars than the usual attention to program efficiency- 
Typical of other wort directed by narr is. a theory of visual light 
source detection developed by Ullnan* Several factors were exeainedt among 
which were; absolute intensity values, intensity compared with the average 
illumination in the scene, both local and global contrast t and lightness 
contrast. A surprising experimental result was that these factors were 
shown to be insufficient for explaining our ability to detect light sources 
in the visual field, even under very simple conditions, While high enough 
contrast, for example, is a sufficient condition for creating the perception 
of radiance, it WBS shown that a light source can sometimes be detected in 
cases where the contrast is very low, A method for detecting light source! 
was than proposed and implemented, based on the comparison of two ratios: 



for a given pair of adjacent areas,, both intensities-ratio and gradients- 
ratio are computed. If neither of the areas is a light source the ratio* 
thou Id be equal. If they am not equal, on? of the areas is a light source. 
The method can thereby detect the source and compute it* intensity. 

A collection Of cur previous papers on vision* learning, and 
representation has just b&en published by He Draw -Hill, under the title. The 
Pilfchblagi; of Comjmttr GJf^ofl, Sales have been encouraging. This Is part of 
en effort to disseminate our work and speed the technology transfer. 

Proitiiclivitjt TBchnologv 

Electronic circuit problems were selected as an initial focus for developing 
automatic manipulation and inspection ideas because the parti, are fairly 
well Constrained to a few types and because the vision is somewhat mora two 
dimensional than in most problem areas. 

Horn's program for visually aligning Ic chips in preparation for lead 
bonding was a step forward since this Job is now done entirely by hunan eye 
in apite of attempts made by industry to automate the process. This work 
was largely completed in Sep tenter of 1&74. Since that time son* polishing 
has taken place, and efforts have been made to transfer the technology to 
practice. An article ob this, "Orienting Silicon Integrated Circuit Chips 
for Lead Bonding," has been accepted for publication in Compute?' Graphic* 
and Image Frocess^nj. GCA, the leading maker of automatic bonding equipment, 
produces i machine by which one person does the alignment for several 
bonding stations, but they have not been able to eliminate the human 
alignment step. The need appears to be particularly acute in the 
manufacture of hybrid circuits which Should not be manufactured by foreign 
suppliers. 

Taenzer has worked on inspection higf-er up at the level of solder blob 

checking. Unlike Horn's prog ran, Taenzer's is still too slow for imraediete 



application, but it nevertheless tackles the problems of practical 
inspection. 

Another advance in doing applied vision was reported in (.ozano-Peret 1 
work an a system called. "PJWAR" which harrows from natural linguap/e work in 
the sense that it generalizes the Augmented Trans it ion Network parser idea 
in a direction thit makes it a workable way of describing real images . 
Loianq's paper shows that the Approach Vastly simplifies the recognition of 
electronic circuit parts which previously was dona by cunfcersome, opaque, 
and less powerful programs. 

notion is another area, of interest Since many applications, in both 
manufacturing and recote navigation require A tracking capability. BpeCkert 
has put together a system that tracks a bouncing ping pong ball. He is 
working on improving both the image handling and the common sense 
under stand in 3 of this system so that it can handle general images, with hiuti 
accelerations , 

Manipulation has continued to progress a? well. Raibert, for example, 
has succeeded in writing a program 6y which an am learns about the dynamic* 
of trajectories ^ Because of recently solved hardware prohiens, his work to 
ditto ha* been confined to three Joints* And more work is planned* 

Work on force feedback ha* gone forward using the older, simpler arm by 
Silver, now improved by the addition of a tilting, force sensitive vice* A 
radial bearing has been assembled autcr.aticaily, in the last few months 
this was complenented hy insertion Of resistors and IC dips into* printed 
circuit cards, (To be sure, special purpose machines can do this, hut at 
great expense, Moreover, we have dene this exercise primarily because 
issues are addressed which SfrneralUfl and lead toward assembly problems 
beyond existing special purpose devices,) 



learning And Debugging 

Soma drugging issues are addressed by Goldstein's work on « personal 
assistant scheduler. The idea of debugging is of central importance, 
despite the apparent difference l fl the selected domain froo others in, which 
debugging IS studied. Two visions of a scheduling program have been 
developed, experimented with, and compared. The flnt was a siople LISP- 
based, brute fnrce program with ne failure end little debugging; it was 
In piemen ted tn serve as a point of comparison. The second version was 
implemented in C0h"N|V£Jt and involved backup alung with procedural 
representations of people end their preferences, The third version, of 
which many nodules are complete, is based on the nor* advanced frames 
concept for representing knowledge and involves ongoing implementation ttf 
Goldstein's Ideas apput debugging and bargaining, fhe version now under 
itudy is to wort in part by simple first pass appro* illations followed by 
sophisticated conflict-resolving debugging specialists. We believe ehU 
could lead to a atuch nor* accurate And knowledgeable theory or project 
management since the problem of scheduling in the face of diverse demand* 
and tradeoffs is common to all hard scheduling prppleas. 

Systems Uork 

We have argued at length that the era of time sharing is eventually going tn 

end because of startling reduction* in CPU and central meaory costs. To 
hasten this development, with its corollary reduction in system complexity 
and software cpsts, we are working on a so -eel led LISP machine. Now ggs of 
the logic design Tor the -draft" version of thr- machine is Complete and it 
will be sent out the door for wire wrapping of the boards inminently. 
Considerable code has been written a; well. 

To be precise, the system will work by way of two levels of 



interpretation; LISP expressions will first be compiled Into 16 bit, high. 
lSYSl "macrocode-" The macrocode will then be translated Into the LISP 
UChlne's particular microcode. Host of the first pass LISP to macrocode 
compiler is now complete. About 3DJt of the Macrocode U ^ :: r a c ode compiler 
Ik also dons. On* version of a complete systen is running Kith siBulatitm 
of tha two compiler* running on the FDPIQ using P0P1D instructions as the 
target Language Instead of the LISP machine's microcode, 

During tha last six months other system work has included th* 
Installation of 2$5fc of nou core memory on our PDP10, As expected, dramatic 
iaprovemant In systea performance has resulted and people are once again 
working with and Learning from large programs. Until this naraory was 
Installed,, really large programs ran so slowly they were beyond practical 
debugging, 

- 

intelligent Terminalt And Vem Large H<tt<f Botzt 

Our work on these programs started in January Of this year and OUT progress 
is still largely in the form of proposals rather than accomplishments.. 
Nevertheless, we have invented and plan to work on several notions. we 
believe to be of fundamental importance to work on the Largs, data bass 
problems, for example, tha ideas of SeilantlC paging, semantic cache memory, 
semantic interleaving of execution and fetching, and the use of the semantic 
difference net idea for searching . 



APPLICATIONS 
ORIENTED 
BASIC 
STUDIES 



APPLICATIONS ORIENTED BASIC STUDIES 

The Artificial Intelligent Liberator? proposes to work Pll Issues central to 
the large data base problem, the intelligent terminal problem, and the 
natural language engineering problem with a broad research program based on 
the following tenets: 

The key ta mixing computers mora helpful ahd tffac'ivr Ltss 
in how we represent knowledge In them, New representation 
ideas have exciting prospects^ Natural Language interface 
engineering is important. There is a deep need to study 
common sense reasoning. And people must be understood as 
well IS the Tiles, 

THE; LARGE DATA BASE PROBLEM 

It ii now possible h indeed common, to record huge amounts of in format ion, 
Unfortunttely there Is no companion ability to use that information 
intelligently. Large riles have becone black holes for knowledge in many 
cat ei. military equipment maintenance records being a particular example. 

netting useful information out of large files is thus an important 
practical problem, U raises important iJteoretiral Usues as well. 
Artificial Intelligence has done well In tightly constrained doneins -- 
Vinograd, for example, astonished everyone with the axpertias of his blocks- 
world naural language systen- landing th^s kind of ability to larger 
worlds has not proved straightforward, however. Even deiling with a story 
in an elementary reader seems to require a great deal of background 
knowledge and Hare sophistication of representation. Handling large 
technical data bases may he herder still, Tha time has cone to treat the 
problems involved as central issues. 



The Large Bate Ease Problem AnJ The Intelligent Terminal Problem Overlap 

The collection of ideas we propose nay be allocated among several headings: 
large data bases, intelligent terminals, and fundamental artificial 
Intelligence research. He stress, however, that there is considerable 
overlap both with respect Cd needs and to potential solutions. To be sure, 
we feel that knowledge representation and organization form the core of the 
large data base problem while natural language engineering, common sense 
reasoning, end user models form the core of the intelligent terninal 
problem, But we also believe that neither a large data base urogram nor an 
intelligent terminal prograit can be fully successful without substantial 
dependence on existing and needed artificial Intelligence wqrfc in both 
• rets , for the aoment we dwell on large data bases and on how something 
like an intelligent terninal system might be used in concert with large data 
bases. Later on fundamental research is discussed In ISSUES ORIENTED BASIC 
STUDIES 

Tf\e $matt {fata ffoj* Frob-le* SuflftJtl Iteas 

Before talking about what is difficult, let us begin with what is easy. 
Some Of what we would, like to dp with large data bases and intelligent 
terminals, has been illustrated aiready on very simple rinnains, involving only 
snail dit a tunes. The work Is laportnnt because it serves to establish i 
content for the discussion of large files- A glance at the better dialogues 
With Winograd's now classic natural language system shows several important 
capabilities already demonstrated in miniature. 



■ The system /\<n sme underslnndiitff 0/ natvrol ivngvugt This 
is iiiinllii, nofaerety desirable for the large filt 

program because establishing an enthusiastic corununity or 
users can be easily prevented by lust a little tog such 
Complexity. 

■ The Jifstom oratcfres /pr particular eytflti It stands reedy to 
alert the user to disturbing combinations. Constantly 
active *demon H programs watch thfl flow of information, into 
and out of the data bass and complain if appropriate- 

■ The system can introapec-l into sent 0/ its owit actiirftfeij 
it can CJtJtfer G/uejUofiS about its own potri structure This it 
Important because we *ill often want a large data base 
system to explain how it arrived at a conclusion or to 
supply additional arguments to the ones already given - 

■ T/lfl iifstent trans lates English descriptions into search 
prog rami This is an important precedent because we believe 
data bases should eventually include English descriptions of 
themselves and how they can bo acteisad and used. If these 
descriptions can be understood and exploited, the problem or 
dealing with diverse data bases can be ccnsHerebly reduced. 
The English understanding required, being specialized to 
data bases, say not be much mora difficult than that 
encountered in the blocks *rorld! 

A natural question is How mch distance lies between these talents of 
Winograd ' s system and something that would be useful to a doctor, 
congrea&person, or economist? We believe the distance is considerable In 



*oma dininsions and *li«ht in atheri. In the next section w explain ^i*h 

dimensions seem critical to ui and *here hb believe heavy rss^rce 
allocation, is required r 

IDEAS AND SOLUTIONS 

AW /de« J/r ftsarejejili/i? ffnouiaitye Have ftcEtfnj JYn J pe c t s 

Questions wist be answered toy finding relevant facta. Finding, and r el evant 
an tha key words. 

«e believe that coping with the problems of accessibility and relevancy 
in large data bases regulr-es understanding of those large data base* in 
term* of hierarchically organised frames, mis it crucial. Cretins and 
using hierarchy in knowledge representation* ti difficult, and thi problem 
has coma t& be widely recognized a* a central dominant theae firmly entwined 
with nearly all important artificial intelligence challenges. 

Ninny's frame theory has traced considerable excitement precisely 
because it addresses the representation problem squarely and provocatively. 
Hare Is the essence of the theory, is explained by Nlnskyj 

When one encounters a new situation [or nake* a substantial change 
in one's view af a problem), one select! from oenory a Structure 
called a rr^Tie . This is a remembered framework, to be adapted to 
fit reality by changing details as necessary. 

A frame is a data- structure for representing a stereotyped 
situation like being in a certain kind nf living room or going to 
a child 1 * birthday party. Attached to each frame are several 
kinds nf information. *ome of this information is about how to 
use the frane, Sane is about what one can aspect to happen nest. 
Coma IS about what to do if these expectations are not confirmed. 



We Cftn think of a frame as a network dT nodes and relations. 

The "top levels* of a frame are fixed, and represent things that 
ere always true about the supposed situation. The lower levels 
hive many terminals -- "slots" that must be filled toy specific 
instances or data. Each terminal CM specify Conditions its 
alignments must meet. [Th« assignments themselves are usually 
smaller "sub-frames. "J Simple Conditions ire Specified by markers 
that fflljjht require a teminal as 5 1 snmfc ;'> t td be a person, an object 
of sufficient value, or a pointer to i sub-frane of a certain 
type, flare complex conditions can specify relations, awong the 
things assigned to several terminals. 

Collections of related f rases era linked together into frame - 
systems . The effects of important actions are mirrored by 
t ran s forma t ions between the franes of a system- These are used to 
make Certain kinds of calculations economical, to represent 
changes of emphasis and attention, and to account for the 
effectiveness of "imagery.' 

For visual scene analysis, the different francs of a system 
describe the scene from different v i ewp o in t s , and the 
transformations between one frame and another represent the 
effects of mowing from place to place. For non-visual kinds of 
f rm6S, the differences between the: frames of a system can 
represent actions! cause-effect relations, or changes in 
conceptual viewpoint, Di fferent frames of a system share the same 
terminals ; this is the critical point that oakes It possible to 
coordinate information gathered frpn, different viewpoints. 

Much of the phenomena logical power of the theory hinges on 
the inclusion of expectations and other kinds of presumptions, A 
franc ' s terminals are normally already filled with * default " 
assignments . Thus, a frame may contain a ureat nany details whose 



supposition l* not specifically warranted by the tltnktirti. fli«e 
have many uses In representing general information, most likely 
casus, techniques for by-passing "logic," and ways to make Utiful 
. generalizations, 

Thi default assignments are attached loosely to their 
terminals, so that they can be easily displaced by new item* that 
batter Tit the current situation- They thus can serve also as 
-variables," or is special cases for ^reasoning by example," or as 
'textbook cases," and often make tha use of logical quantifiers 
unnecessary , 

This frame-system are linked, in turn, by an inforrtatlpn 
retrieval network When a praposed frame cannot be nada to fit 
reality — when we cannot find terminal assignments, that suitably 
natch it* Seminal marker conditions -- this network provide* a 
replacoment frar.o. Thes* inter-frana structures maxe possible 
ether ways to represent knowledge about facts, analogies, and 
Other Information useful in understanding. 

This theory is a scholarly integration of soma old ideas and many new 
ones. It opens the *ay to research on previously intractable, questions. In 

particular the problem of establishing a correct context Is exposed as a 
problem to work on rather than a problem to worry about. Hinsky Integrates 
and elaborates several approaches in considerable detail : 

■ The Idea of clusters of frames organized like places on a 
road map around dense local paths, larger and less dense 
arteries, and Finally the limited access superhighway*, 

■ The Idea Of group charaqSarltatlon by multiple prototypes, 
d-capitols as he calls them. 



■ The idea of moving from one hypothesised frame to another by 
way Df difference analysis, the so called similarity not 
Mn. 

■ The Idea of preserving as auch Information as possible- in 
moving from frame to frame by knowing invariants properties 
of terminals audi iUbrrames, 

Elaboration of these is important to the development of entwined 
representations capable of enabling fast access to large data bases., 
Fahlman and Grossman are working OP other weclianisms which nay well prove to 
be important prerequisites. They ore motivated by i feeling that 
intelligent systems should have a very fast, perhaps even parallel mechanism 
for working with semantic netnorJti r They argue that questions like these 
Should have nearly instant answers: 

Which ships have bad boilers? 

Are elephants long nosed marsupials? 

Do penguins flyT 

Fahlman' s progress report offeres a solution in the form of a theory of 

nodes, links, and propagating, labels. Ta* theory was developed after 
Fahlmar. tried to apply frames to large problems and Stumbled into * 
fundamental Set of problems: symbol mapping (the problem of finding tha 
properties of an object whose description is in the form of a tangled, 
multi-levered fraaeh inconsistent fact detection anti the enforcement of 
restrictions; and of finding the right frame during recognition. Falhisan's 
hardware nets., described in an appendix, ara one possible solution to these 
problems which certainly oust be solved before frames can be applied Co 



large data bases. As ye t the ideas ore new ta US and Ma have FID solid 
feeling far whether a. Fahlman machine would be i break-through or a 
disappointment on the order of the resolution-based theoren prover of the 
put. or the per ceptron -based "learning machines , N It is clearly important 
to find out. 

We believe tflflt all Qf the frame accessing ideas will be Increasingly 
necessary (and effective) Hith increases In data base size. As yet, 
however* the frame idea is largely untested. Many examples have been 
suggested to illustrate how frame ideas Tiig.it be applied, but no working 
system yet demonstrates their real strength, 

Ms have seme evidence that frazies can be used to structure laru* 
amounts of scenario knowledge of the sort involved in children's stories. 
Creating such a system for a problem of considerable real world importance 

i 

is a high priority objective, Ue> have not yet devised franc structures that 
would capture such scenarios as nations in conflict. We imagine that there 
could be a frame theory for the participants in international conflict 
scenarios that would have terminals and defaults for friends, enemies, 
attitudes, traditions, desires, wealth, and occupations. (Abelson has 
drafted interesting caricatures of such representations, and leaves open the 
problem of filling them out with "lame data base" detail.) Such a line of 
work would hopefully lead to systems c&pahJe of hypothesizing conflict 
frames early enough to activate the mechanisms that could help avoid their 
consequences. Having such structures should allow frame instantiation, for 
exempli, from the *'e«J rorJt fluei news sumnaries, in the sane way that 
birthday party scenarios could be instantiated by stories from children's 
readers, as suggested in Charniak's work. We hasten to emphasize that 
dealing with something so complicated as the fttv VefJt Tines is very bard and 
oust be considered long range. Vet some of the problems of representing the 
desires, moods, and likely actions of people and countries are suitable for 
direct work now. 



Frama Structures SuggtH Mttworjf Qrgvwiivtiim Schtms 

Ira Goldstein has pointed nut that frane theory lead* to the notion of 

semantic cache memory, Selecting a f rafflfl by hypothesis, bringing Chat frame 
into fast memory, and leaving it there until dislodged by newer frane 3 is 
pood in the saaa way that ordinary cache memory is good. Once referenced. ■ 
frame is likely to be referenced again soon. 

It Is equally truo that frames closely related to • frane being 
processed are likely to become relevant. Consequently the idea Of pagingi 
comes to Bind, Sema ntic paging would group together seaant ieally related 
information units that tend to be accessed togsther and could be used to 
bring possibly relevant information up to at least the fast secondary 
Btgrase level. 

In addition, we suspect the frame hierarchy, similarity networks, or 
Other frame link irtsi nechanisms should 6e used to automatically .bring, 
particularly attractive frames out of semantic pages into the semantic cache 
while processing proceeds- Thus, expectations wpuJd be used In an overlap of 
semantic processing arid fetching functions analogous to the old idea of 
overlapping the executing and fetching activities in ordinary machine 
1 ns true t ion p r ote s s i n g . 

Suppose, for example, that a large data base system is asked to supply 
information about world oil price stability. Pealing with this, the system 
would follow a thread from the oil frane to the international meeting frame 
instantiated, to specialise it to 0-FEC gatherings. By virtue of a semantic 
cache, Traces encountered along the way would be retained- By virtue nf 
overlap between semaritic processing and retching, other frames would be 
brought in dealing with pricing policies, recent geopolitical events, 
environmental, and othflr related issues- 



Natural LanpuoBG Interface £flii\teeris\g fs impo rt<r n I 

^presentation of knowledge, is said, befere, u tlw nost vital 'part of our 

proposed focus Oh large data bases, But other topics strongly related tp n .. 
intelligent terninal focus deserve discussion because in our vie* they can 
ba studied most affectively by a synergistic program that deajs with both 
large data bases and Intelligent terminals in i unified way. 

Natural language understanding is on example of a technology that seems 
vital to both arrorti. It is fortunate, therefore, that this is one area 
*hflrt wa fael hard work will surely produce good result* even though some or 
Hi* desired scenarios will require considerably more natural language 
strength than did Winograd's blocks world task. There ire now many natural 
language systems with may flQD tf ldH1 . The field seems ready to gel; it is 
■ tint for integration. 

It seems clear, incidentally, that natural languor,? should not be 
limited to accepting questions in English -- to be really useiul, the system 
should accept questions in English, should generate natural English renarts 
and reports, and should be able to extract information iron largo English 
filas as well as highly structured files. 

In particular we visual i*e bringing together something like Pratt's 
LINGOL or Wood's Augmented Transition Mat grarjnar or Marcus' new wait-and- 
see grammar {described in an appendix) on the sentence fragment level, 
Filrmre's and Hartin's case analysis on the sentence level, and fchank's 
conceptual dependency theory on the semantic frontier. 

An analogy with progress in computer vision research nay help clarify 
why we believe rapid progress can be made toward wrapping up wort in natural 
language research at the level of natural language syntax and elementary 
semantics. On entering a completely new, unexplored area, the first Step 
must be to get a feeling for how to describe things, usually mating 
simplifying assumptions that place interactions In the background, Guzman's 



wrly work on analyzing polyhedra is a good example. 6y looking at many 
samples, he developed an *<f «ec theory of how regions should be grouped Into 
bodies In thia early phase, It was appropriate to focus narrowly -- the 
way shadows interact with other sort* ar lines mas justly avoided. 

During the ne*t stage of research in the drawing analysis work, 
descriptions were refined. This made the results af analysis more precise, 
but analysis became more complicated because enrichment means alternativea 
and alternatives of ten lead to ambiguity and subsequent search- It Is 
proper and natural In such circumstances to tnUrate inerritient search- 
oriented strata a its for using knnwledo* so as to exposo, through experiment. 
whore the (toy knowledge problems really are. Later on, however, as 
descriptive apparatus becomes still richer, the interactions becoae so well 
understood that alternative can be explored not throng* classical search 
but rather through defernent of all decisions until choices can be made with 
complete confidence and total coumitment. Nalta 1 work represents this stage 
In line drawing analysis: first lines are described in a rich language 
capable of noting lighting characteristics as well as a line's physical 
cense ~ shadow, crack, boundary or whatsven second, constraints, anna the 
lines are so well understood that possibilities Tor individual lines type 
identification can be carried forward until each untenable choice can be 
logically eliminated- There is never any backup, 

syntax analysis seems in a healthy state just under the final stage - 
note and mora evidence suggests that the natural ambiguities that occur in 
language should be handled not by guess and backup but by a wait and decide 
strategy, This in brief is the thrust of some maturing work by Marcus, an 
educed araduate. student. His progress report on this subject, included as 
an appendix, has a comfortable feel. The parser described already handles 
the. following rather complicated sentences in the sense that it separate 
them into noun groups and establishes the role or each noun group in the 
case structure. 



Yesterday, Jolln gave the book. to the girl. 

Who did John give all of the books to? 

Is there & hoy sitting on the bench? 

John wanted the boy to be given a book. 

The boy John wanted to persuade to sit on the bench hit bin. 

The system makes the vital structural distinctions between 

They are Tailing rocks. 

They are breaking rocks, 

Which are not at all Similar SCmantically , The system also handles these: 

He believed John. 

John was believed by US, 

It was believed that John was till. 

He believed that John was tall, 

He believed John to have been till. 

John was believed to have been tall, 

Tn< boy JohEl wis believed to have hit gave me a bonk, 

The bay John was believed to have been hit hy gave me a book. 

Has the man sitting on the bench' 

Has the nvan sitting on the bench welting- to the store yesterday? 

Is the girl John saw sitting on the bench walking to the store? 

Is the girl John saw sitting on the bench? 1 



People fltijt Bt tfnderalood As Veil As The Film 

Eventually systems busE be intelligent not only about the information they 
contain but also about the people Mid use them. Like respected assistants, 

they must tnoH the shape of what we want from- brier instructions if they are 
U supply us with what really concerns us as well $i with what wb 
specif ically ask for; they should not trouble us with unreasonable demands 
for describing subject detail or stupid Questions about how much we already 
know, only if they understand us to that degree can they get the message to 
Garcia. 

SupposQ, far esample, a known user steps up to a machine ond asks for a 
report on Turkey. That should be enough, in the sense that good default 
presumptions aiaad at knowledge about people, should handle decisions about 
xueh questions as these: 

■ Is the user taJung abau: th& bird or the country? 

■ Where does he lLte to got in forma t ion Trom T The 
Encyclopaedia? TJle *ew Yaft. Tines? 

■ How much information does he usually want' Two pages' 
Twenty? An abstract? An annotated bibliography? How is 
the quantity decision related to recent evidence or 
interest? 

■ Presuming the user is talking about the country, in what 
particularly is he likely to be interested * is he normally 
interested in information about Soviet border issues, U.5. 
aid, heroin production, the Dardanelles, rugs, Cyprus, or 

What? 



If the user asks to be kept informed in the event of t crisis, then the 
question of what he 15 interested in .becomes two dlnensianal: the system 
■ust knew not only tha topics that ars to be monitored but also what 
constitutes a crisis. The fact that sons U.S. tourist flits Jailed for 
buying dangerous drugs Is not likely to be a crisis to Henry Kissinger but 
it might bo far a local eiioassy gfficiai, The Secretary uf State presumably 
wants tn hear about how the Turks are likely to react to changes In the U.S. 
aid flow, while the fellgw who worries about tgurist problems probably need 
not be awakened each tine new infornatioEi on such subjects appears. 

Proper monitoring can be handled by development of a theory of user 
■odels and a theory of semantic interrupts. For exanple, such a theory 
■lght involve a technique by which interrupts would be generated whenever 
speciTic. user dependent frames are activated by tentative hypotheses or 
confirmed by a complete analysis. 

All of the work on "demons' 1 derived from PLANNER and CDNJJIVF-R work Is 
relevant, of course, but we need in addition an opistomology of people and 
what they know. We believe this Is strongly related to the other side of 
the system, the machine, its large data bases, and what they collectively 
know. In particular, we believe it win be possible to use the same frame 
oriented knowledge representation theory far organizing knowledge about both 
people and the dgnains that interest them- It should be very productive to 
keep this in mind., especially since the ultimate system cannot be had 
Hithout significant progress on modeling both, however similar or disparate 
they may be. 

Naturally users should have easy, smut-t^i se-cnanisns for resetting the 
knowledge the system uses to decide when ta bather them. Probably most 
users will require something between absolute importance threshholding, and 
constant information rate, independent of how auch action there is -- 



lonetlunQ like what we get an the finincial pages. 

Sana relevant work is already underway to handle the problem of people 
and their preferences in the personal assistant context. ATter til, even to 
arrange a person 1 * schedule Some facts about when he likes to get up, whom 
he likes to sec, when he can tolerate interruptions, how Important he Is 
relative to various superiors and esijjloyees, and the like mist be known and 
used, AS additional nodules are added, particularly in the dimension of 
automatic note digestion and information retrieval, wore progress in this 
direction Hill be nr-.--.vi. 

For a major large file program, hoover, additional professional help 
seens or preeminent importance, 111 this we are fortunate to wort in close 
collaboration with people having deep interest and considerable experience 
in thinking about how people think and what they kito*. He can consequently 
propose a broad approach, on* dealing bath With the artificial Intelligence 
issues end soma of the human modeling Questions. Indeed we believe that a 
synergistic approach based on such collaboration is far more likely to be 
productive than separate, divided assaults. 

Several Qomltts $*m Suitable far The Saiic faitorch Wort 

We want to look at the activities of various professionals including the 
following; 



■ The Equipment Ha in tenan c a Officers. This of course Is the 
prototypical Large file user. Direct attacks on existing 
files say cloud some issues at the start, but as a domain 
for study it is Interesting by virtue of tha potential size 
of the data base, the need for trouble oriented watchdog 
demQUi, tht possibility oT using common sense qualitative 
reasoning, and the need to maintain sensitivity to user 
suggestions about Just how to watch and monitor things. 

■ The Strategic Materials Aftalysists, The idea is to worry 
effectively about reserves of various metals and other 
strategic materials. The problem features both hard tabular 
data and English phrased opinions about political trends, 

■ The Logistics Planners. The Vast prDbVen of getting things 
Where they should be and tracing blunders derserves careful 
attention. Part of the problem is a resource allocation 
problem with conflicting goals. 

Again we stress our opinion that all of these areas have much in common 
In terns of the scientific challenges they offer, 

SOME QUESTIONS 

Why Do We toltcanirate On T?xt F\i?$? 

We do not, really r We believe that the text examples we have given cover 
the Bain issues of semantic organuaeton that all targe data base problems 
present. Certainly considerable thought will be given to nore structured 
file* is well. We favor selection of target domains in which eombiiutluns 



Of text and structured data naturally occur, An example Mould be 

maintenance records in vtiizh fr&e text and highly constrained forms are 
blended together. 

Hoa Luff* Is Lvrge? 

We believe the hierarchically structured, frame oriented approach we propose 
Will lead to s-ysteas that perform well or large data bases whatever the 
size. Larger date bases will require mnre muscle In the knowledge 
structure, but not in. linear proportion, &y adding a layer or two or depth 
to a hierarchically organized data base, hig* increases in the ii« of the 
information covered can be achieved using basically the same theory uied at 
higher levels, The aain obstacle then becomes that of understanding the 
■aterlal fast enough to prepare the lowest level description frames as it 
000* flying bv. 

At this time 50 little has bean done we cannot realistically speculate 
on the size of the data bases that will be handled in the near and middle 
term. But since the knowledge organisation research tre propose seen* 
fundamental, we feel It will be central to handling any data bases that can 
be handled at all, 

THE PROPOSAL AND THE MILESTONES 

It i» clcer that nark on large files interacts very strongly with other foci 
of laboratory interest. While it is not possi&le or appropriate to draw 
sharp boundary lines into the picture, it is possible, however, to guide 
both new and existing resources toward a large file focus, Thii point of 
view i£ reflected in the following elan ftlementa- Target dates range from 6 
BOnthi for creation of a structured flats base with natural language itiput to 
30 months for implementation Of semantic paging and similar ideas. 



■ Implement i structured data basfl witti natural language 
access for a small sunset of the relevant statistics Tor a 
particular strategic material or equipment naiTitenance data 
base, This represents a capability already achieved for 
others smaller donajns- fcccnmplisrmEn t of our goal here 
will ba marked by being able to direct simple statistical 
queries in English to the data base. 

■ Concentrate our natural language effort in the intersection 
of Che problems raised by Gur personal assistant focus and 
large file issues. After t*D years tine He would expect 
good natural language front ends to a variety or personal 
assistant and large data base nodules* 

■ Push forward simultaneously w lth the network taxonomy wort 
of Fahlm&n r expecting a Ph.D. in June of 1976. This is to 
contain ideas and axperinenlal results defining the possible 
rule bf a Tast, well conceived A-KIUD-OF processor in the 
larye file program. 

■ Implement a frame sV&tem representing interpretive knowledge 
about the data base. This interpretive knowledge consists 
of both common sense facts such as the fact that oil must be 
refined as well as expert knowledge such as the title and 
costs involved. This miles to r.e re pre seats the application 
of existing theoretical ideas to an application area. Ibis 
allows queries to go beyond simple statistical inquires -- 
data hases can be asked about trends, possible dangers, etc. 



■ ImpUocnt a demon capability for real-time monitoring of 
critical situations, Demons Mill monitor both symbolic and 
statistical additions, to data bases. We must solve 
technical problems which cone up when demons are activated 
by complex patterns, and ha must solva problems of 
computational efficiency needed when aultiple demons are 
activated, 

1 Extend the natural language capability so that a system Curi 
add statistics to a data base by processing text. The 
system is freed from the tine consuming' and error prone 
process of hunan reading dT textual rta:a and human entry of 
Che relevant facts- Fur the mo re facts can be added more 
quickly and the inFornBti on flan - csr have less tir-e luu- 
This of course is a critirai issue far data bases 
representing estimates of the supply of critical defense 
items. This milestone Mill he achieved when a systoo reads 
news articles on a sanple material . 

I Use of frame organisation and especially hierarchies and 
differential Trama painters to allow semantic paging, 
semantic hashing, Semantic pattern matching. It is. this 
organization which orps fcnygnri trad-.iicinal ideas that we 
expect to allow reasonable computation times Hven on very 
large data bases. 



■ Keep these diverse research activities coordinated and 
integral the results into a pilot 5*5 ten B! rapidly as Che 
ideas, reach a sufficient stage or maturity. This would be- 
the duty of staff researchers commencing work in early 1976. 
The r9£tflt is to be a vr.;i docum^ntoU system continuously 
available for experimental use as several of the natural 
language systems are now. 

HARDWARE 

It is conceivable that a uacftlne inspired by Faklman's research on knowledge 
structure night becone part of a successful abroach to certain aspects of 
the large knowledge base problem. In gur opinion, the Fahlman concept is 
the first hardware design that promises to handle information problens for 
*bich the Classic Serial Von Neuman architecture Bay Ultimately prove 
inadequate as the total foundation on which to build a truly intelligent 
system. Wo do nut propose ta begin the design of such a machine now because 
the Idea Is not sufficiently mature and it Is not sufficiently substantiated 
by limitation. After this work is done, however, we hope it will prove that 
the design of a machine should be in a future proposal. 

Meanwhile it is clear that developnents in artificial intelligence- 
problem areas will require v^ry heavy demands on computing resource*. 
Experience shows that very Urge address spaces and dedicated central 
processors are required to cope with the complexity barrier, At the moment 
the LISP machine concept h proposed for development in a separate section h 
seems well suited to this purpose- 



TECHNOLOGY TRANSFER 

■We plan to worX With a view twarij providing implemented nodules with both 
the enchantment appeal and the inherent usefulness required far esport. We 
Turther plan eariy involvement nf people from a selected user community, and 
we realize that the availability of s«ch involved users is an important 
consideration In danain seleetion. 

PERSONNEL 

At least the following people would contribute to a. large Filo effort. 

Negotiations are underway to recruit other artificial intelligence 
specialists from outside the N,I.T. comunity. 

■ Professor \PA GOLDSTEIN will be the project Coordinator, He 
has been leading the laboratory's personal assistant work 
aimed toward understanding the scientific questions involved 
in the development of a system capable of scheduling 
appointments and events, organizing nptcs aivi abser vat ions, 
and retrieving a wide variety of knowledge, FJt£D ItERN Is 
currently helping Goldstein ^n, the scheduler nodule* 

■ Professor NARVIN rtlN^RV Hill wprfc on the problems of 
Knowledge representation and comnon sense reasoning. It is 
expected that flinsty's frama theory will be of considerable 
importance, tn studying the problens he win be assisted by 
several graduate Students. 



■ SCOTT FAHLNAN and RICK GRQSSHAN both came independently to 
the view that sone hatter approach tg retrieving knowledge 
ll demanded by the current problens fating artificial 
Intelligence . They are working hard, on the new network 
label propagation idea to handle the sorts of .scenarios we 
See Coning up in the large file cpntent- 

I Professor 51 v hour papert is interested in the question of 
arga.n iza t ions > the mu-Jiis-s '.hat exist within tliem, end the 
problem of supplying those managers with in format ton that is 
dt ones useful and desired. Such questions will certainly 
tie important in the intelligent terminal context ilnce one 
function of an intelligent Cerninul wou^d bfl to facilitate 
the interaction of an entire office full of people. 5 7 ■■: ■-• F 
R0$£Mt3f,RG and Jin STftMSFIELD will assist Fapert In helping 
to identify the genuine Hit Ita dene nt in format ion issues and 
use these in the process of selecting * target domain wt-tb 
which to- work. 

■ 

■ Professcr VAUGHAN PRATT and his Students are moving Forward 
rapidly with the developnent of UNGOL, a very efficient 
natural language systen facilitating griimnar writing- This 
IS expected to compete favorably with Wood's Augmented 
Transition Network formalism for the. prize of being the best 
Sisple system for imp lenient in? natural language front epds, 



■ CAUDACE BULiWIMtLE, DAVID NcMMALD. and HITCH KARCUS are 
graduate students specialiiing in natural language studies. 
BullNinkle t» working an a module fir interfacing various 
pergonal assistant Facilities to users, NcDaiiaLd is wurMno 
on knowledge based dialog, and Harcus is finishing a 
powerful new parser believed to integral* the best feature* 
Of a number of separately davelaped systems. All together 
WO iee this work moving toward an understanding of the 
relationship between task size and, complexity on one hand 
and system sophistication on the other, Natural language 
interface GnginaBring seems plausible now. 

APPENDICES 

FAHLrTAHf! A BVSTEH FOR REPRESENTING ANS USING REAL WORLD UNDULEWE 
MARCUS: WAIT-AND-SEE STRATEGIES FOR PARSING NATURAL LANGUAGE 



APPLICATIONS 

ORIENTED 

VISION 

AND 

MANIPULATION 



APPLICATIONS ORIENTED VISION AND MANIPULATION 

The Artificial Intelligence Laboratory proposes to work on applications of 
computer vision and manipulation with the emphasis shifting from tool 
building toward solving red! problems, using well-known, commercially 
Important problems a* a source of challenges . in particular, we plan to 
devote part of our effort to a hard assembly problem Involving putting 
together or taking apart devices with flexible components, awkward 
geometries f and poorly constrained tolerances. We further plan to broaden 
our activities by inclusion of work on tracking moving objects and on real- 
world Inspection, these problems how being within reach of solution > 
Success on these new problems will lead naturally and smoothly to progress 
on some aspects of the general problem or applied image understanding, 
although not all of them a* we argue later when we discuss the future of 
basic studies in vision. 

Productivity TecJmolejy tfaj An InfiQrtanl fiution 

Broadly speaking, the overall goal of pur work in applied vision and" 
Danlpulatlon ii to develop uses- for artificial intelligence technique* In 
hand or Within reach now. We have this goal because we want to do thfi 
following thing* : 

■ Halntain the economic strength of the country in the 
competitive international arena through automated 
production! particularly in assembly- 

■ Create a mora flesible brand of manufacturing capable of 
very rapid response in times of rapidly changing need. 



■ Prepare for automated uini:iy, both under ground and undersea, 
a* well as automated farming, space exploration and 
recovery, ordinary maintenance, and maintenance in hazardous 
environments. In part this oiim creating an autofioaous 
vertical guidance technology Cftpabla or coping- With 
unprepared environments . 

We belirve thest objectives art Vitally important because a p r e enti n e ji t 
computer science and industry is oi\w of the major Strengths of the ftnitt-4 
States- Ve mitt translate this advantage into car re* ponding capabilities in 
areas uAere ovr mrld pvsitivn is erorfinp, especially (a productivity. We 
think changes will come about quietly When a certain threshold of technology 
is passed over. Just as pocket Calculators Street the Country like A tidal 
wave, we believe computer vision and manipulation will soon sweep through 
the manufacturing and maintenance industri&i, tut before discussing hhat 
jooft neans, let us loot at the path on which Me have been on and how it 
reaches the Stated objectives. 

■ 

Progress In Transferring Artificial Intellittnct Ideas To industrial 
Practice Involves Four Major Steps 

Ve measure progress in terns of where wa are in executing the following 
plan; 

1 Pick the Simplest possible domain, do vision and 
manipulation in that domain r and use the results to measure 
the difficulty oT the problems in order to allocate 
resources and predict when applied results should emerge. 

This has been, done, of ccurse- Th* blocks world was the doaain; the 



difficulty of vision dictated the concentration of effort on it and the 
relative neglect of stipulation. It becane clear that the existence of 
global knewledne and the non-linearity of meisa effects render nost previous 
wort on image processing of Little vain*. Instead, problem solving, 
constraint exploitation., and symbolic low level processing became central 
issues is discussed elsewhere in our discussion oT directions for basic 
studies in Vision- It also became Clear that vision Is Very hard, and for 
the sake of dear term application results, the goals in vision research 
should be divided into no-tricks basic study and no -ho Ids -bar red, special- 
purpose, domain -da pen dent work* 

I Dnce the problems are uncovered through suffering in the 
trenches with thi "simple" domain, then it is tin to get 
together hardfr&r-a and software teals capable of ceaningful, 
cost conscious prototype a r ran gene* t to solve particular 

problems. 

This toe is behind us. Our so-called Hi ere automat ion Laboratory exists 
and supports applications wort. It is described in brief later on. 

3 Use the tools to teckle real problems. At first these may 
be selected because they are amenable to solution through 
the tools and techniques in hand -- obvious undisputable 
commercial viability is a secondary objective. 

Our wort on electronic assembly was chosen for suitability in this 
phase. Hany results, described later, arc in hand, Ug believe we will 
complete the remaining work in electronics this year. 



4 Finally, unleash the technology on the most serious problem 
Where the saving would be greatest in human and fiscal 
terns. 

This is to be our primary purpose during the contract period. A% the 
moment, this purposB translates to th& assembly of mechanical device* 
exhibiting all the tough assflttbly problems wt can find which seem essential 
td Industry and apreprtate for solution by some combination of vision, force 
sensing manipulation, and now understood Artificial Intelligence programing 
ideas. 

■ Industrial exposure is of course vital to a program like 
this where sensible resource allocation and technology 
transfer are 40 important. V* 4X4 therefore fortunate It 
BIT to have 4 very active Industrial Llason Office which has 
helped Arrange for us 4 wide spectrum of contacts in 
industry. 

Production facilities visited Include those of Texas Instruments^ Data 
General, Digital Equipment. IBM, the RCA hybrid circuit facility, the 
KennecQtt Copper Company Peabody coal mine. General Motors car asseobly 
plants , Oelco integrated circuit facilities, AHP incorporated electrical 
connection and electrical 4sseably facilities, the Foxboro Industrial 
controls radlity. United Shoe Company's automated electronic equipment 
■aehitiery production, 4nd the OCA facility charged with the production of 
advanced lead bonding equipment, Nora of this industrial contact is 
planned. In particular e nucber of advanced production facilities and 
research laboratories will be visited in Europe this fall, again to be 
financed by the MIT Industrial Liason Office. 

We will return to the subject of hard probleas in a Boment* 



THE FOUNDATION FOB THE FUTURE 

We now give a brief description of the Laboratory, mova then to e brief 
discussion nf soma prpgrass made with it, and then finish with tha proposals 
for the future - 

In our last najor proposal, we enphaslze-d two opinions: 

■ He argued a need for a workable*, inexpensive set of hardware 

and software topis fpr doing research in applications 
oriented Caaputar Vision and manipulation. We laid put a 
plan for the creation of these tools- 

■ We suggested that tool creation without one or two target 
dnmains can be inefficient and leads to popr resource 
allocation- We specified the problem of electronic circuit 
card manufacture and repair as suitable for bringing general 
issues into crisp, nanageable form. 

All of this work is well along as evidenced by our progress report for 1979* 
The inexpensive, exportable vision and manipulation laboratory now exists, 
It has demonstrated its usefulness by supporting important target domain 
projects involving electronic circuit cards and by facilitating first steps 
outside electrpnics in other worlds of interest like the world of small 
mechanical devices. 



rfrt Hffrtfwere Is Ready 

Wb have a PDF 11 NO with extended instruction set (E13K 31K, of core memory, 
two 1.2 megaword di-Sk. drives, anij a CkT'SO graphics display terminal. In 

addition, We have the following devic&Sr 

■ Two VlCARK mechanical arms, 3/4 human size, with six joints 
and six degrees Of freedom, position and velocity sensors in 
most Joints, and gripper hands- These conprise our primary 
manipulation system. Texas Instruments, the University of 
Illinois, and the Naval Research Laboratory hive ordered the 
Mir designed! arm and didte aro expected to follow suit r 
Figure 1 shows the general appearance of these eras. 

i A Vldican picture digitizer, consisting of a television 
camera and a; television monitor, and a digitizer which 
converts the scanned picture to & hit digital data at a 
maxioum rate of 65- microseconds for each point in the TV 
scan. This is currently our primary vision system, 

■ A Hughes module tab 1b laser with an output of .001 watt, used 
principally us a jointflr or troLukir.g devize. 



so. 
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I Two motor-driven Birror-de flee lion systems, by Spatial Data, 
Systems, Inc, Each mirror systen has txo independently 
controllable mirrors with perpendicular axes of rotation , 
and settling tines of appraxinate ly 3 nil 1 i seconds for 
notion oT the mirrors. One systen is used to deflect the 
HkigheS laser; the other is attached to a PIN diode Via an 
optical system and is used for scanning scenes as ft Slow hut 
very linear , high resolution vision input device. 

■ An Optronics International,. Inc. photovriter 1 capable of 
naking extremely high, resolution photographic negatives on 
fila from stored digital pictures - 

■ A precision X-Y table, nade by the Icon Company, capable of 

notions in two orthogonal directions with an accuracy of 
,001 inch 3 over a range of motion of 6 inches with a maximum 
velocity &f 4 inches/second. This device 1* used to 
position work under the Bucnanieil arm or the vision system, 

■ An Analogic tilgital-to-inalog and analog-to-diflitil 
converter with B D to A channels and 64 A to D channels, 
capable of performing data conversions in 3 microseconds, 

and to which He have added an additional 16 D to A channels. 

The So/lbwre J* Revdit 

The Productivity fechnology Group PDP11/4A uses standard DEC software N the 
Disk Operating System (DOS}, in an effort to mate software developed within 
the group useable elsewhere. The group has developed the following 
software' 



V I PIN Is the visual input prograa, It Is used ia. 

Conjunction with dither the Vidican or the PI K-diode /mirror 
system ta obtain digitized pictures which are stared OH 
disk. ThB user can select arbitrary sub-sections of the 
entirB picture being Planned, and the program continually 
displays to hin the sub-section of the picture ^hich lie has 

selected, 

RAPPER is a pro gran for cnn.pu.ting statistics an stored 
digitized pictures, MAPPZR can provide histograms and 
intensity gradients, printouts of the raw data end oven 
graphic representations of the picture itself. Operating In 
conjuction with & program that runs on the GT40 graphic 
display terminal, it can threshbold a picture Into 6 
intensity gradations, and then cause the picture to be 
displayed on the uT4Q, ftAPPfcK enables the user to examine 
small subsections of the entire digitized picture for 
detail, or to combine subsections together to get a larger 
view of the statistical analysis of the picture, 

LISP, u fuilj-Qeneral LISP interpreter, with shallow 
bindings and a non-recursive garbage collector which 
conserves nenory. This LISP does have the restriction that 
atom print-nanes must be 5 characters long,, ar less. It 
does standard MS file I/O, and can additionally da input ar 
output free a variety of terminal devices, or from the 
Artificial Intelligence Lab's PPP1D timesharing system. 



■ RUG, a symbolic, DOS-oriented debugger, which replaces the 

DOS debugger OfiT. Using RUG. a user may debug standard BOS 
format .programs; the debugger makes available to hid til 
his symbols, disassembles the contents of core into assembly 
language statements with symbolic references included* and 
allows hi* to enter assembly language code into core by 
Merely typing It In. RUG has various numeric and special- 
purpose type-in and type-out routines (such as ASCII and 
RADIK-S0) L It allows the user to Set breakpoints In his 
pragrac, and additionally allows him to nonitor the contents 
of a care location, register, or device for some desired 
condition. 

I A set of arm control routines for the VKAHfl nechanical am. 
These are implemented in LISP, and Include routines fer 
moving Che arm Co i particular state-vector (essentially a 
position and velocity for each Joint), and Tor halting tha 
arm's notion and holding it still. The routines incorporate 
a trajectory planner which decides on the route the arm will 
take to gat to tha desired state-vector, and a dynamic 
routine. The dynamic routine leaks at the desired State' 
vector as computed hy the trajectory pLanner at the end of 
the next time increment, and computes the torques which must 
ha applied to each Joint in order to achieve that State- 
vector. 

Finally, there is an improved editor which Utilizes the graphic 
capabilities of the OT40 terminal to implement a reel -time edit mode, and 
there are programs for transacting Tiles to and [Yon the timesharing system, 
for displaying images on our GT40 termininal, end for other typical ays torn 



activities. 

Artificial Intelligence fleaearcft Feetfa Protiuctivitv rvthnotegp Development 

Hie flow of basic Artificial Intelligence techniques to applied problems has 
begun In the Artificial. Intelligence Laboratories and a complete list of 
achiBVBQBnts in this direction by the community would surely include the 
impressive work done in Japan on A, I, inspired £11* code readers and circuit 
card inspectors, the *orlt At Stanford on manipulator control, and our ami 
work, of which tlie fol Lowing are nf particular relevance; 

■ Inoue and Silver demonstrated the automatic assembly of a 
radial bearing with \Z micrometer tolerances, thereby 
demonstrating that ccwipifcffJstf gaenbiiei can oe made usiny 
flexible 1 1 programmed manipulator* , The achievement further 
demonstrated that kraouledpe based JolCt ttltsitive 
manipulators are the kvg to advanced asStatbtii operations, 
and that high I wire 1 proaramiiag lartjULdte-s mike oro( rarani. if g 
for mechanical assambta a tfidJfrUe C C n-C « Jf t - The hearing li 
illustrated In figure 2. 

■ Hnrn itiowEd how to apply the technique nf knoaledae based 
profile analysis, i)*rrawffd ["ran earlier blocks world wort by 
BinFardu Shlrai and Others, to the applied problem Of 
orienting tC chips in preparation for lead bonding. 
Figure 3 shows some typical profiles encountered. Hat Hason 
is doing. Che same thing with autoaiatad diagnosis of 
electronic signals of tho sort that technicians look, at on 
oscilloscopes. 
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■ Taanzer uses reptoji ?roariflji t&ttz to Inspect solder Joint] 
for the standard defects — pin holes, cold jpints , end open 
holes. See figure 4- 

■ Speckort has begun to incorperate Mflihe^ let/el fcjiouJerfje 
about obstacles In low level tracking routines that follow 
objects through Otheruisi difficult acceleration J , Figure 9 
shows a trace of Spetsert's points or observation as his 
fystem tracks a ping-pong ball through a bounce. 

THE CHALLENGES TODAY 

Using our now well developed tools, we plan to accomplish the following; 

■ We plan to conclude our ARPA sponsored; automatic assemhly 
program^ This will bo amplified in a moment, 

■ We plan to daal with a difficult, important, and costly 
inspection problem, solve it, end, use the resulting solution 
to determine If cost effective solution is in sight. Via 

propose the inspection of jet engine castings as the 
candidate domain. 

■ We plan a new program in the dynamic vision of tracking and 
vehicle navigation, We believe this i*ill prove to be a fine 
source of new important problems suitable for replacing our 
work on assembly as that phase, of our program goes to 
successful completion.. 
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fosemblit OJ A Gasoline Engine Add rents The Town Issues 

It aeeua to us that we should be workino an elements from the overall list 
or problems that are best fitted for us in terns of our own expertise and in 
terms of what workers it ether institutions are doing well enough. Ub 
therefore propose to build the snail gasoline engine -- very likely the 
NOdel airplane engine i Unit rated in figure 6 -* with a view toward somng 
the following problems: 

1 The programing problem, Our intention is that the assembly 
program should looH like an annotated parts Hit With advice 
about the assembly, not low level instructions, W are 
inspired in this effort by comp l emeu tary work at Stanford 
and hy our own history of success with very high level 
languages like PLAINER and COM I VI R. 

■ The parts orienting problem. The parts are to be scattered 
at random, on the table or drawn from bins. The SRI results 
will be adopted, if sufficient. 

■ The flexible part profclum. The engine has a gasket, We 
hope to connect up the fuel Una as well. He believe our 
handling ef flexible wires in our electronics -assembly work 
will be valuable experience. 
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The two-band coordination problem. The engine has a 
difficult problem inasmuch is piston, piston pin, and 
connecting rod nust be brought together, Inoue'i and 
Silver's work with the radial beiring, presented in the 
progress report, gives us son* solid close-tolerance 
manipulation experience, 

The Straight coordinated movement problen. In an Hffnrt to 
eliminate the complex equations normally required for 
Seal In g with em dynamic s,, Raibert is constructing a model 
whereby the arm controller learns, froa previous attempts at 
movement control end stores this learned behavior into a. 
large data base on disk. Tha first step is to cause slow 
notion of the arm between pairs of points in order to teach 
the system the gravitational parameters along the path in 
between ■ Then the im controller can begin to learn about 
the dynamics flT notion along the sane paths by iterating the 
motions at speed, trying 1 to learn about the arms dynamics in 
Order to radu.ee the gap between the previous iteration and 
the desired motion. 

Coordinated vision end manipulation. Insertion of the 
crankshaft into the Connecting rod offers this Challenge, 
The try will not win every time, we suspect, bringing in the 
opportunity to deal with fault detection and recovery in a 
natural way. Spetkert's tracking went may be useful here* 
aa well as Taemzer's inspection and insertion wort. 

Tools, The entire Job will require some serious attention 
to thought about the manipulator 1 * tools.. 



ft Inspection. Ue nay, if fortunate, be able to test the 
assembled engine and even Adjust the needle valve. 
Inspection for surface- defects, however,, is to be done In 
the casting domain, described later. 

CaJtinja Seen To 0e Tfte fliffbt Qpm\H For Vintal Inspection Stupes 

The visual inspection of castings is to be 4 Step toward thi development of 
practical visual Inspection systems. Figure 7 illustrates the sort of image 
produced by a pinhole defect, one of nany types to be studied. In our 
laboratory, we have been developing tu« technical tools needed to understand 
images of complex surfaces like these. The current work of ttarr, In 
particular, offers iniight into what to do because it eJaaLs expertly with 

the prnblEia of texture. 

In order to become aware of the practical Issues associated with the 
Inspection of metal castings, Woodhao has visited the Advanced Casting 
Laboratory of Pratt & Whitney Aircraft and the Draper Division, foundry &f 
ftoctwell International. They have been extremely interested and helpful , 
They have been not only a ready source of defective castings but they have 
ALSO provided Uoodhas with the detailed specifications by which various; 
parts are currently inspected. These inspection standards provide the 
appropriate benchmark for assessing the performance capability of visual 
inspec t ion s ys terns . 

The tfflrtontit "'lafofl 0/ Secf-Oriejitins And Tracking Is ffeodfl for Studs 

Over the years , the problem or seeing and understanding static scenes proved 
(0 hard that anticipated work on the vision of motion remained out at 
pasture with few important results energing f/ron the Isolated attempts that 
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were aa.de. While there are still nany difficulties, the tine has coma to 
look at notion problens seriously because we hava tetter ideas about 
representing visual knowledge; because we have better hardware and software 
tools, and because the need plainly exists. 

To this End j hb plait two efforts, One now underway as a doctoral 
thesis project attenpts the following scenario i 

■ A Vehicle Is flawing an or over hilly Mars-like terrain. The 
Vehicle wants tu keep a goed model of the ground it la 
moving over with a vi&w toward tracking its position on a 
nap If It has one or toward making a map if It does not-. 

The self orienting study will address two key scientific issues^ Tirst 
the issue of dealing with representing situations evolving through, timet 
second the issue of exploiting constraints evidenced only by comparison of a 
sequence or Situations , not by single situations alone. 

As tha system looks at the world, it is to nake a symbolic network 
description OF What it Sees consisting of nodes representing hills, hill 
features, and tno T- joints th&t at- pear where one hill obscures tha 
background boundary of another hill further behind. The nodes are linked by 
relational pointers with labels like IU-FROKT-OF „ TAL-LEtt - THAN , PART-QF t and 
jso on. Figure 4 Illustrates Tha changes in these descriptions through 
time* It is believed, will b« subject to handling in terms of Hinsky's 
theory of frames, the now maturing theory of network description, defaults, 
•contexts, expectations, symbolic transforms, subgrouping, and nore, as 
described in apfLicatiqxs omextsd basic sro'flies. Each symbolic description 
of one instant in tine will be one frame in a collection. 

As tha system composes its frames, it will use constraints devolving 
from motion parallax and from changes in the observed vertices. By SO 
doing, the notion becomes a positive contribution to analysis, rather than a 
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zoma llcation to be relieved, This aspect of the work has- strong antecedents 
In Waltz 1 work on static scene analysis which showed that by properly 
addre-ssiny the constraint question, shadows and the influence of one vertex 
on. its neighbors- became terrific aids to analysis., not handicaps. 

Another problem, to some extant the inverse at self or ten tin, 9, i& that 
Of tracking coving objects; 

■ Constrained objects are moving through Ah image. Knowledge 

of the constraint is to be used to Jielp trick then through ■ 

situations involving tenporary interruption* of vision, as 
when one object obscures another; and involving fait 
changes, as when a ping pong ball bounces off of a paddle; 
and involving objects constrained to move together, as when 
vehicles follow one another. 

The study in tracking will build on the hark of Speckert appearing In 
the progress report. Speckert has tracked ping-pong balls through bounces 
which would baffle- most tracking algorithms by combining simple world 
knowledge with ordinary adaptive mechanisms. Bore of this will lend itself 
to trackers good enough to, say, track people or to allow vehicles to follow 
one another in convoy, as in a nine or on Jlars, 

TECHNOLOGY TEANSFER 

We plan several steps which we believe will considerably shorten the tide it 
would otherwise take to bring our vision and manipulation wort into practice 
on the factory riaor. The work seem too important to depend on the slow 
osmosis that would occur without deliberate planning. Far the most part, 
these transfer plans involve spinoffs froii our norcal academic duties which 
are done with no cost to our research contract. 



■ We have Introduced a new HIT course on the subject of 
computer vision and nanipulacion in order to prepare new 
graduates for active leadership in the field* A text has 
been Undertafcon as 4 mechanism for encojruging such courses 
el sb where* 

■ We have introduced a summer session course for the J1 IT 
industry oriented summer program There wa* a favorable 
first-year turnout with engineers and managers from Boeing, 
Honeywell, Mew Yarft Telephone, The U, S. Department of 
Transportation, Uestinghousfl, Kodak,, atid Xerox attending. 
General Hotgrs has purchased our Videotape introductory 
Artificial Intelligence course - 

■ Wo plan articles for the trade literature describing the 
laboratory equipment and summarizing What has been done with 
it so far. This is to complement our normal publications in 
the scholarly journals. 

■ We have whenever possible built the vision and manipulation 
laboratory out of easily obtainable aqulwent and we have 
avoided modifying this equipment- When new equipment has 
had to be designed, we have avoidsd one-of -a-klnd; ln-house 

Construction. 

PERSONNEL 

i Vision and manipulation work will be done under the general 
direction of Professor JEATHOtD H, P r HORN, well Known for 



his work, on machine vision. Including Horfc on shape fnaa 
Shading and reflectivity analysis. 

NEVER B1LLHERS Will be in charge of system software 
development. JOHK PUHBRICK will do mechanical design. 

DAVID SILVER and TD1AS LDZANQ-PEREZ are working with DAVID 
TAE.NZER, and NARK RAIBERT toward achieveti&nt of the engine 

assembly . 

BOB k'D^DHAH will work on the visual inspection of certain 
castings. HARK LAVIW will ^p the dynamic vision of *olf- 
orlontttlon^ GLEN SPEdHE'RT will cenjj-lenent that work with 
his fen owl edge -based tracking effort. 



ISSUES 
ORIENTED 
BASIC 
STUDIES 



ISSUES ORIENTED BASIC STUDIES 

The Artificial Intelligence laboratory proposes to c&ntinue i basic studies. 
program aimed at dealing with deap, central Issues. The Issues ara to be 
exposed by "hrort in computer vision,. CQtttnoji sense reasoning, and expert 
problem solving, The ubiquitous representation problem is involved in all 
three areas. 

THE NEED FOR BASIC STUDIES 

■'All research lies on a spectrum between pure development and pure research. 
Host work, including all of ours, lies far from either of these estrones. 
Still, one can either select a need and bring tools to bear on It or one can 
select issues and choose the bast domains, in which to eapose thea. The irrork 
described in this section lies iwre toward the issues oriented focus t where 
It Is harder to specify timetables or even to predict accurately what Mill 
happen, Historically some of our laboratory ' s greatest contributions have 
cone cut of work like this that can be aimed but not tightly steered. But 
this Is not to say that cur basic research is done by guess and lurch. Qur 
long range goals ere clear, they are described in sections devoted to 
machine vision, COimon sense reasoning, and expert problem solving. 

All three of these sections emphasize the importance of sound 
representations, We ftet -f eir* that proptr rtpr-?i*i\i;QtiQir ts tfte ftey to 
edtfanced vision, common «*&# reaionine, eni exptrt proiEem solving, /tut e* 
it is to mejitf otfler aspect* of Artificial I nteiiijeace. Indeed i subtitle to 
this section, ISSUES OtiEIEttTEQ BASIC STUDIES, could be, ASPECTS OF Ttti 
ttEFREEENTATIOit PROBLEM, 



SEEING AND UNDERSTANDING WHAT IB SEEN 

The long range overall goal of basic studies in vision ii the creation of 

human level ability In computers that dan understand, describe, and search 
far things in images. Td the ext&at that processing visual Information 
requires intelligence^ our studies, of vision help us to understand nore 
Sflneral Issues. We have and will, continue to It am ? lot about p rob Ism 
Solving in general by studying vision problem in particular. 

It must fia understood that the overall goal of creating human level 
visual ennpatence has. proved difficult. At first the critical path subgoal 
•seened Id be the very proolen si'.ving is.i.ies wc* nr-w xnow ouch about, New we 
f eel h an the basis of considerable experience, that tho hey is the 
•Xtr&Ction OT" as ftuth Information as passible using relatively knowledge- 
free techniques, Tho purpose of this section is to defend cur point of View 
and to braak the critical path subgoal of iaage-to-symbQllcdescription 
translation into milestones and forecasted dates. 

Creating Computer Vi&ia* l& Hard 

The study of canputer visiao began very early in the history of artificial 
intelligence. Originally the problts of seeing and understanding what is 

seen was to be a tsstbed for working out deep problem solving and prograo 
organization issues. Eventually this testbed purpose was served, but only 

after overcoming many frustrations and solving many unexpected problems - 

I Even blocks HOTld configurations produce complicated images h 
confusing prcgra^s wjt'n niitja', : 1 l'.ininat ion effects, 
indistinct interior edges, »is Wading scratches, the large 
variety of verte* configurations, and so on- 



■ Image- Input devices like vidicons and Linage dissectors do 
net produce the Simple clean images one might suppose -- one 
oust understand and account for variations in intensity 
sensitivity across the image, errors in image coordinate 
information, electronic noise, light source hum, and lack or 
dynamic range. 

Vision, it was discovered, is very hard. 

We fleve Scratched Tht Sur/ace 

To establish, a contest for what wft propose, we list a row major steps 
forward in. vision research.- This is raet intended to be a complete listing 
of achievements pv*r the years — progress reports are the proper vehicle 
for that. Instead this is a sketch showing the shift in approach that has 
resulted froa our progress. Vision is no longer naively thought to be a 
metier of a saart p rob Lea solver looking at a few salient image points — 
instead It is clear that knowledge re presentation, constraint exploitation, 
end care with low level vison are all critical. 

Moreover, it has become clear that the low level visual processing we 
talk of will Lead to techniques that demand a great deal of computational 
power. Thankfully IC technology is progressing very quickly,, so that a real 
time, flexible visual processor is becoming a technological possibility. 

Given the research described in this section, it should be practical to 
look at photographs of buildings and describe then well in five years. This 
research into basic issues that confront general-purpose visual processing 
will complement what we are doing now by way of special purpose vision, 
which has already begun to produce the prototype level results described 
elsewhere in this proposal , 



He Know Thvt Conttrcint exploitation Is The Right way To Deal With Some 
Problem Of fl.nu 2^3 ii 

Wilts' work on understanding 4Cen8S surprised everyone. Previously it was 
believed that, only a program with a complicated control structura and lots 
of explicit reasoning power could hops to analyze scenes like that in 
figure 9. *low we know that understanding the constraints the real world 
ioposes on how boundaries, concave and convex interiors, shadows j and cracfcs 
can cone together at junctions Is enough to itake things such simpler. A 
table which contains a list of the few thousand physically possible ways 
that line types can come together accompanied by a staple matching program 
Are all that is required. Scene analysis is translated into a prcblera 
resembling a Jigsaw puzzle or t set Of linear equations. No deep problEp 
salving effort Is required; it is Just a matter of executing a very simple 
constraint dependant. Iterative process that successively throws away 
incompatible, line arrangement combinations, The implications of this type 
of analysis are far reaching, not only in vision, but also in natural 
language. It is now apparent that the important task of dateraining the 
role of a noun group in a sentence -- agent, instrument, destination, 
conveyance, or whatever — can he done by Just such a constraint process 
exploiting properties of word order, prepositions, individual verbs, and 
global context. The Idea is a Ley elenent or Marcus 1 wait-and-see sentence 
parser, mention ad in APPLICATIONS OfUil'TtQ BASIC STV&IES and described in an 
appendix:. Even common sense electronic circuit reasoning has been shown by 
Sussaan to use a siallar process. His work is also described in an 
appendix. 
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We Knew That Adders terrdi it j rJie Wfllnematicj Of Illumination Emtblti The 
Creation Of Imparl ant A/wliliU Program 

To understand the rEal world, we must have a different sat of primitives 
from the relatively simple Lin* trackers suitable and sufficient for the 
blocks world, [n the past, image intensities hive been used primarily to 
segment Images using ad baa methods based pn differences In average Image 
intensities or some higher ordar measure. This is not pjoed enough. In 
order to deal With prahlens in inigfr analysis it is, important to Understand 
how Images are formed. It ti not sufficient to try myriad processing 
techniques borrowed from other fields in the hope that one can bo found that 
does something useful . 

Instead on* should realize that images carry a great deal of 
exploitable information about the three-dimensional nature of the surface* 
of the object* Imaged. Horn has demonstrated this when ha shewed how to 
unwind the differential equations of ioage illumination In a way that allows 
Shape to be recovered from shading gradients for staple reflectivity 
functions. Later on. working with Hair, Horn gen&raliEed a notion conceived 
by Land a.nd showed that the natural reflectivities in a jumbled array of 
colored papers can be determined despite complication* from uneven 
Illumination which varies across tha inage. Such wort must he continued. 
He oust develop an understanding of the nature of our visual Invironnent and 
the image formation constraints if we are to successfully model what ii 
being seen. 

We Xnou Knoiffierfje Ii An Important Adjunct To Inane Proc«jiJtS Techni^v.9i. 

Figure 10 illustrates the action of Shirans line Tinder, representative of 
the work applying problem solving and control techniques to image level 

Vision. Its fundamental thesis was that knowing a lot about the objects in 
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a scene makes it possible to analyze the sc«ne with primitives that 
otherwise would prove inadequate, 

Shlral'S program knows, for example, that the objects it worts with are 
typically brick shaped and produce many parallel lines. It alao knows it 
oust expect aligned cbjflcts stacked on one another. Such facts enable it to 
guess at tlie location and direction of faint interior lines after it has 
trecid out the strong background lines, Understanding the world should help 
process real images as well: Knowing the approximate place and nature oT 
some Clear features like the doors and windows in a rood should help locate 
walls, clocks , pictures, and so an. 

Shlral argued that his system illustrates some rule* of program 
organization end style known popularly as "thfl heterarchiai approach," 
Lately the development of heterarchiai philosophy seams to have reached a 
natural plateau with the work of Freuder. His thesis, subaitted in June. 
1975, develops heterarchial control structure ideas to new levels by 
presenting a collection of ideas about how syst-eos can isake effective use of 
Suggestions and advice: a hammer head, for example, proposes a possible 
location for its handle, Such systems have been much talked about and 
sometimes implemented in simple ways for simple situations, but Frauder's 
result ia a theory rather than an isolated illustration- Freuder develops 
such Ideas to the point where it sbucts unlikely that further progress can be 
made in the prablera-solving/heterarchial control direction without vastly 
strengthening the processes that translate image intensities into the 
sywbolic assertions required by any sort of reasoning program, 

We believe mare toor* OB COfltroE structures /of i/fiion mill not nwte 
jense until our undcrstaAdiAf 0/ lev level symbolic prc-Mssittf catches up. 
The real world differs from the blocts world significantly because there is 
a need for ouch better analysis between the Ion level feature analyzers and 
the high level problem solving. Ue turn to this next. 



We £r«J Th$% The Translation Fr^/n Intensity Arrniri To Svrnboliv ToposraaJtu 
Hep& Is Far Hart fmflOrtdflt Than Has Original Ijf ffceuffJH 

To step out of the blocks world Into the real world require* more than 
problem solving, heterarchy, high level constraint,, understanding, and 
Isolated image processing techniijueSr Past progress in artificial 
intelligence is a necessary but not sufficient foundation for supporting 
inner* understanding systems capable of analyzing end describing pictures. 
We have cone to understand that human quality vision requires talents not at 
first anticipated- It is these that we propose to wort on now. 

Our central thasis is that if we are to understand how to make 1 
maeninc see, we must get our hands the whole way around the problem not 
Just the so-called "high level' parts of it. We know this in part because 
afTorts to rely on the high level wort have not produced satisfactory 
results, whereas new worh on low level vision by Karr and his students, dene 
with A view toward mating each level in the analysis as self-contained as 
possible, has sparkad excitcmcr.: . These results and the proposals that grow 
out of then fom the core of our planned work on basic studies in vision. 

A Solid Theory Of Vision Is Cominf 

Hare, in synopsis, is how Harr feels about vision; 

■ The function of early visual processing is to compute a 
description of the grey-level changes present in ah iaage in 
terns of a vocabulary of gray-level change primitives. 
These prioitlves consist oT straight contour segments of 
various kinds (SHADING -EEGI, EXTENDED-EDGE etc,), LINES. 
BLOBs. and of various parameters bound to them such as 
FUZZINE5S. CONTRAST or LIGHTNESS. POSITION, ORIENTATION, 



simple measures of their SIZE, and a specification of their 
TERR 1 UAH ON points , This prlaltlve description Is. obtained 
froo the intensity array by kna>>l edge- free techniques, and 
It is called the primal sietsli- The Illustration In 
figure 11 hints, at the density of the information taken from 
the Image and represented in the primal sketch. The prinal 
sketch differs from an array af feature points in a subtle 
Kay, KhiCh IS explained fully in flarr's appended paper. 

From our ability to interpret drawings, one nay infer the 

presence in our perceptual equipment of symbolic processes 
that are capable of grouping Lines, points and blobs 
touethar in various ways. Man-symtialic techniques, like 

examining the power spectrua of the spatial Fourier 
transform of the drawings, cannot account for these grouping 
phenomena, since the groupings are performed by mechanisms 
Of construction rather than necnanism Of detection. 

For aott laages, the primal sketch is large end unwieldy - 
However, it can be capably analyzed by a oechenism that has 
available the symbolic processes discovered in the previous 
stop, together with the ability to «lect itens. out of the 
primal sketch on the basis Df simple first nrder 
discrimination* acting on the principal parameters. Hence, 
it is argued h texture vision rests on grouping operations 
together with first order discrininations operating on the 
primal sketch h rather than on operations on pairs of points 
in the ir: tensity array as suggested hy J u 1 e s z . It 1* 
further argued that the set of processes whose existence is 
necessary tn order to explain our ability to interpret 
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drawing* is also surricunt, when applied tQ the primal 
sketch, to explain the range of texture vision that Is 
present in humans. Fourier and power- spectrum techniques on 
their own are certainly deficient, and probably also 
unnecessary. 

Certain fish, incidentally, can nnmic their surrounding onvironpent 
very well velth an adaptation time of only a fev seconds. Harr's texture 
theory has lad to a theory of the control of this texture camouflage ^ 
Presumably it depends on measuring and Ditching the small number of global 
texture statistics that Harr believes defines texture space. This theory is 
being tested elsewhere at JLI.T. under the direction of J, Y. Lettvin . 

■ The extraction of a form from the primal sketch using these 
techniques anounts to figure-ground separation. Except in 
difficult cases, this extraction can proceed successfully 
without calling upon higher level fcnowledgi, and it precedes 
the description of the shape of the extracted forn- 

Thii has two Important consequences. First, the isolation and delivery of a 
fore to subsequent processes does not depend on being able to assign an 
accurate high-level description to it; and second,- because of this it i* 
easy to compute rough descriptions of complex forma. This ia prohably 
essential for the fluency of subsequent shape analysis. 



■ The extent to which higher level Knowledge acid purpose 
influences the processing up tp this stage Is vary United. 
There is at present no reason to believe that higher level 

knowledge is needed to cpnpute the primal sketch at ell; 

Its role in tha extraction of form frcii the primal sketch 
often can be United to deciding which fora should be 
extracted. 

It is conjectured that in ell cases , higher level knowledge need be only 
weakly coupled to the processes that separate figure end ground. This 
relegates tha use of higher level knowle-dpe to a ouch later stage than is 
found in current machine vision programs, and simultaneously confines its 
impact to influencing control, rather than Interfering with the actual data- 
processing that is telling place lexer down- 

Karr's appended progress report explains the theory in more detail. 

Pro j re j s In Vis f on Involves Seven t Giwli 

■ Complete the implementation of the low* and intermediate- 
levels or the visual theory outlined above. Rewrite 
grouping code for the sake of speed and assemble the whole, 
large collection of routines into a single package- The 
grouping processes developed so far involve an interesting 
mixture of local analysis, together with an ability to 
generate one or two more global image variables. 



Devise and Implement a theory of lightness computation that 
worts after the extraction of cwtours. Various notj vat ions 
exist for this* partly difficulty In doing it at retinal 
level, partly the feasibility of doing it it this stage 
provided tfiat (as now seems possible > the image is divided 
Up Into regions using knowledfle-free techniques. 

Increase the- dimensions of the ima^e that we cap handler At 
present, Our images are ]£0 by IZ9 elements, which 
corresponds to Viewing an area 1 Inch square at a distance 
of about five feet with th& human retina. Even this small 
Image stretches our resources, and prBCBSSina one sucn linage 
cm take 20 or Jfl minutes of CPU time, If we spend 
considerable erfort an re-writing chunks of the low- level 
code, He could probably move to a 216 by Z56 image with wily 
a small added cost in processing tine. 

Put BLO-Bs securely into the primal sketch. K BLOB li a 
snail, indivisible patch whose lightness dirfers from that 
of the surround. Although it is clear that BLOBs should be 
a primitive ale&ent, the existing techniques for da tec ting 
theo from the output of the orientation- dependent analysis 
are lemewhat elementary* 

Devise a theory of the alignment or an axis to an extracted 
forta. This 1$ vital for the ideas we are currently 
developing about the transition from two to three- 
dimensional descriptions. One subgosl here is to davisa and 
implement a theory of symmetry dfltflction because this is one 
way of providing an axis. 



■ Carry out experiments to discover whether the theory 
provides a complete account of texture discr lmination 
abilities, This is a difficult and arduous undertaking 
5lnce we are really searching for a counter-example . But 
analysis Of (say) a hundred Image*, chosen for their 
awkwardness* would go a long way towards establishing 
adequacy. At present about ZQ have been processed, and not 
all our texture -process ins facilities have been used. 

■ Assacile a comprehensive report of the. complete lpw-level 
visual processor. It would Initially be published as a 
Technical Report f then a published monograph. 

b'e Vtih to cxpmss in t,Vc stroieeJt possible terms our l^OIS that it is 
unreal Uttc to expect useful resuEtJ jrom most comport* it t J 0/ 9 phota- 
tnterpretalion project vnt,il at least tfrU level o/ I o»er- E eve J 
techJIdtOfflcaE competence has bean achieved- We cannot help but feel that 
those who believe all parts of this intensely difficult problem can be 
solved using existing techniques and machines have an Insufficient 
understanding of the magnitude of tha issues that are Involved, On the 
Other hand h we do feel that a careful and energetic basic research approach 
to the vision problem, in which one addresses the Issues that arise as they 
be cone accessible for solution. Is the correct long-tern scientific 
approach. Problems will not he solved JUS-t because one wants to solve them; 
they also have tq he ready for solution. If we pursue the fundamental 
Issues with diligence and vigor, then we nay reasonably expect that within a 
period of about five years, some of these Yery advanced problems will Indeed 
ccas into development, rather than speculation range. But bear In mind that 
this cautious slateaent is in fact extremely radical, and would hive to have 



been regarded AS wildly over-optimistic as little as one year tgD L 

flojt 0/ Our Stated O^ectl&es Will ftejiHra Tvja TtQFs 

Burring unforeseen delays, He should be able to complete much of this 
proposal within two years r Marty of these problems new require labor" more 
than ideas, And unless we have overlooked something important (which Is 
possible), <m should have cocpleted the low- and intermediate- level studies 
by the end of 1977. 

There are, incidentally, some problems we would like to solve but 
cannot approach y»t. These problems include those of understanding the 
visual qualities of gloss, glitter, shinyness, wetness, metal, reflections, 
brilliance, transparency, and transliicence. ¥e feel that many of these 
attributes in an image can be detected using simple autonomous techniques, 
but as yet our grounds for this are speculative, restihgj mainly on the 
intuition that they generate recognizably distinct # qu*lif iars" in a 
perception, father like color does. Ullman's study of fluorescence 
strengthens this feeling because he showed that en autonomous, fcno-wlarfge- 
free Method can detect Light sources. 

Thinking Abtmt A Vision Hnchiat [i A fantrnl KcJiim-Tem GeW 

Once a systen has been implemented in a single, autonomous, package, wa shall 
be able to study higher-level (3-D) issues in the context of what such a 

system can deliver from raw image data. The sheer computing power that Is 
needed to arty out texturu processing on an image of reasonable size means 
that we should begin thinking about designing same special -purpose hardware 
that is powerful enough tp carry ouc the processing In a few seconds. 
Accordingly, we have begun preliminary design studies on such a machine, and 
we wish to continue these- The two prerequisites for serious proposals for 



construction are (1) a firm demonstration that the P&P-1& implementation of 
the whole processor performs satisfactori ly; and (11) that our design 
studies result in a solution to the hardware problems that lies within what 
is technologically possible, and which villi perform the whole operation 
within 20-10 seconds; and ( iii > that the expense Is reasonable and nothing 
like the multimillion-dollar parallel machines of the past, Unless 
unforeseen difficulties arise, we expect to have a concrete design proposal, 
together with proof of its processing adequacy and computational prowess, by 
early 1977, 

COMMON SENSE! REASONING 

It la not possible to fe*l comfortable with the problem of common sense 
reasoning, Progress has been aide- by Bus stain, Goldstein, And others like 
McCarthy at Stanford and Kewell at Carnegie-Nellon* but sone very basic work 
remains to be done, 

Since Slagle's Integration prog ran helped establish artificial 
intelligence as a serious science, researchers have mined out more and more 
narrow domains by ferreting out what kind of Knowledge is Involved and how 
to use it. A correct Complaint rests an the observation that one prograa 
contributes to tha next only on the level of ideas and methodology^ 
Something better IS needed. Collecting a group of specialist programs 
together for a particular kind of problem is good and helpful but not a full 
solution. 

Thinking about common sense reasoning seems to be the right thing to 
da. This is not n new theme, nor is lack of a better theory an obstacle to 
the first round of progress on the very Large Data Base problea or the 
intelligent Terminal Problem. On the other h^nd, we believe that without 
basic research in this araa, progress on practical problems would step short 
Of a full solution. Systems Would tend to be constrained to the specific 



chosen implementation domains with transfer to other areas regaining 
difficult and time consuming. A systen Marked out for medicine would be 
useful to the creation of a system fgr ship taint an ance, but much new work 
on the epistamology of ship lift in tenancy would have to bs force fed. 

But What Is the Alternative? Vt bttttvt that UJlrf^rs twrarfiiTj COrttflOfl 
J*rt50 reaJPffiftfi f* jqece^saJY btfore aysteprj Curt fcc created wAich fetfTff 
tatitti about specialized preblem solving dounitn. Trying to- wort out 
mechanisms that draw direct analogies hptweer. heart disease and tho 
maintenance of bailers miuht be one approach- But to us. a better idee is 
to discover how both might be understood in terms of the epistemolngy of the 
aimple, everyday physical world. We therefore be-lle-ve a Aty need t& a 
theory cj coworr ieirse reasoning about simple pn-usitil mechanise. 

Sussman SOd Goldstein have made a good Start by working out notions 
dealing with process: wo now have a better vocabulary for talking about 
goals, plans, and the structure of programs. What we need now is a 
eomplonentary effort focused on our physical environment . We need to have a 
better vocabulary for talking about objects and materials, we need to apply 
our Classical methodology to the Staple physics' world interaction prtiblcn- 
We need to establish what kind of knowledge is needed, what specifically is 
needed, and how much. Va need to worlt out the eplstemology of interacting 
things -- things pushing one another, the idea of filling, filling to store 
♦nd filling to inflate and filling to the bursting point as in hearts end 
boilers. Our systems need to know about the ideas of repair, patch, and 
breakdown, the toughness of common materials, the idea af floating, rolling, 
mating, helping- 

The observations of Jlartin at PITT's Project HAC and of Schank at Vale 
reinforce the idea that specialized reasoning is enabled or at least 
facilitated by analogy to Simple physical happenings. Everyday language is 
replete with physical world terns and usage applied to abstract fects and 
processes . In the mental world we talk of ferreting out Facts; in flartin's- 



social world ownership moves fron place to place With language suited 
equally to Objects; in the logical world facts support one another In a Hay 
not unlike Wlnograd's blocks; and for a typical expert prehlem solvlfifl 
domain, It is rare to think of learning electronics as a child without 
thinking about resistors In terms of narrow wat^r pipes and capacitors In 
teres of water balloons or tome such reservoirs - 

Given such a world theory we believe we will be in much better shape to 
handle all sorts of diverse fields. At first the system may net conclude 
for Itself that a heart can be understood in the same terns used to 
understand the atomic vessel and fluid mechanism, but if we can tell the 
systen such things, much needless re instruction and narrowly focused coding 
would ha tlrtun vented, 

Tnera Are A tfamber 0/ Geetf /Yecedesia For DcvBLoi>ln$ A rvteorji Of Common 
Svjiia RaaSoni raj 

■ The Ph.D. theses of Sussman and Goldstein probed thn 
eplstenclogy of processes, showing that considerable 
regularity esergos when the right descriptive apparatus is 
available. Sussnan worked with the automatic debugging of 
blocks world notion primitives « la WtntJigrad, and Goldstein 
handled simple geometric figure programs. 

■ Sussaan, Brown, and HcPepsott's work on electronics carries 
this thread forward, and gives us considerable understanding 
of procedural aspects of the problem and of the interfaces 

between complex parts- 



1 Schicvt's conceptual dependency theory, although a bit 
evsrfoeused pn the human body, is also an j-ptellaetual 
precursor. Tha important feature is the demonstration that 
convincing bra-ad-based formalisms are achievable, We aspect 
our theory to be considerably mora complicated, but not 
impos&lbly so, 

■ Papart uses the tarm "Qualitative physics" tu cover the 
problem of understanding the pendulum and the gyros cope In 
simpler, push- and- pull terns. In his NSF supported work in 
education,, lie has worked out ways of Looking at these things 
that ^111 be helpful - 

We Plan fo dttac* Th* Coramta 5tr\&e titaiimiflS Problem From four BirectivtW 

In summary, the long range objective of our work is an understanding of the 
Utopia physical interactions of staple physical things sufficiently good, to 
serve as an analogical foundation far programs that want to reason about 
mors abstract problems. (For full power, the analogy process eventually nay 
want to ba capable of working between two specialized domains, as well 41 
back, to the fundamental physical one.) 

Wo believe this is deep research and soma break throuoTi requirements 
prevent us fron giving honest estimates of advanced perfornance lava Is and 
dates. We da believe, however, that there are flood ideas about the next 
steps and that work on these in the two year tine frame will leave us ready 
to make the predictions that would be specious now. 

In particular, we feel that the next steps must be directed to work, on 
these four subjects; 



1 Freiling wants to understand tha prloitivti that are 
prerequisite to understanding basic machinery, He talks In 
terns of shaft? , surfaces, wedges, ana a do^en or so other 
siailar primitives. He further mentions interactions lllte 
rolling, sliding, and the like. And putting everything 
together he wants a system that can understand how a camera 
works by drawing analogy between light and fluid flow, 
realizing in consequence the need for a container for the 
flip, 9 conduit for the light, and a part to control its 
"flow." This is anhitious, and two years' work may not 
produce such an advanced scenario, but the direction seems 
correct and some analogical reasoning ability seems certain ► 
Freiling' s notes on approach are appended- 

Z DeKleer's study of qualitative reasoning complements 
Freiling's by attacking the question of how to deal with the 

boundaries that segment complicated problems into simple 
pieces within which Straiflllt forward mathematical treatment 
suffices. Reasoning of the sort BaKleer is getting at 
pervade our thinfcing whenever we mist have mora than a rough 
idea about hott a compLlcatttd machine worts -- surely it is t 
prerequisite to design and must be worked out if machines 
are to be designed automatically. 



3 Richard Brawn probe-s the general question from a third 
direction, that of concentrating exclusively on using 
analogy in reasoning. His approacn Is Just now taking shape 
■.net now it can be said only that solid geometry may ba tha 
denain because a great deal is known about solving tough 
solid geometry problems by using 3implirifld plain geometry 
problems together with analogy. This domain Kill be pursued 
only if it remains promising as a domain in which to study 
the general analogy problems. The geometry, per it, is of 
secondary interest- 

4 Kulpers li working on representing the kind of knowledge 
that peopio have in their aental naps of cities. This is 
the knowledge which is used to represent where things are 
with respect to each other, how to get fron one place to 
another, and what kinds of groupings they fall into- His 
research will shed considerable Light on the issues or 
imagery, visual mercury, visual problem solving, and similar 
issues involving the internal representation of spatial 
knowledgOr Thar* is considerable psychological evidence 
that for humans ttm mechanisns used for representing and 
manipulating such knowldedge is extensively reused when 
dealing with sore abstract topics like the world of 
organization!. Kuiper's project complements not only the 
program aimed at dealing with comiion sense reasoning but 
also the work of Lavin which is oriented toward extracting 
information from scenes with hills and lakes. Latin's. 
proposals are described in APPUCAT10XS ORlEffTEB VIS1VM AMD 
HAtUetttATIOH. 



EXPERT PROBLEM SOLVING, LEARNING, AND THE 
EPISTEMQLOGY OP ENGINEERING 

A long range goal of Artificial intelligence research ts to nate a systen 
which can assimilate new knowledge and learn from experience. In reaching 
Tor it we have seen iiany attempts at producing universal formalism. lach 
of these has made tha fundamental assumption that knowledge is additive; 
that means can be round for assimilating new knowledge without overhaul of 
the existing knowledge structure . People who design problem solvers »r& not 
alone in making this fatal assumption; many Studflnti believe that they can 
learn by just listening to Tacts thrown at them in lecture halls and that 
learning can happen without much intellectual effort. The truth is that 
learning involves great effort. Introduction or a new fact nr concept into 
a knowledge structure often precipitates upheavals and dislocations in that 
structure. Any system which can assimilate new facts oust bo competent to 
modify and manipulate its existing knowledge r 

Engineers do this constantly, they combine, analyze, debug, and explain 
Stuctures in the course of design. They decide haw simpler structures can 
be combined to achieve partiCu ldf goals and they predict the behavior of 
coopositB structures by combining the more primitive behavior of tha 
substructures out of which they are formed. Then they use analysis because 
proposal of plausible designs may lead to designs which do not quite work. 
This is discovered by analysis and leads into the debugging phase nf design. 
Finally,, an engineer BUSt 19 able is explain the devices which he has 
designed. An explanation is often i description of how the behavior of the 
composite device cen be attributed to the combined behaviors of its parts ► 



Ye Vant To tinder-stcmi FroorvmiBji la fermi ffj Enaiaetrtng feaigm 

Since seme knowledge tt inherently procedural, and thus is best represented 

as pron;raB3 t we are forced to the conclusion that a learning program must b« 
a. programmer -- it must understand knsuleigo represented as procedures, so 
at to be able to uodify and patch then to incorporate desired changes, 

■ Consequently we oust have a sound theory of programs and how 
their structure relates to their functions as defined by the 
goal* of their designer. We are led to a study of the 

epistenology of the design of pro-grans. 

Can we write prograas that conbine. analyze, debug, and explain other 

prograns? We have begun to p but He do not know how to properly describe a 
progran and its relationship to its d&sign- Programs are hard to document 
in clear and useful ways. In order to write programs that reason about 
prcgraas we (Bust lOBehow learn ta describe programs in terms of the 
designer's goals and plans and the interactions Of the parta r The research 
we propose to do Is allied at clarifying these concepts. 

We balieve that some general programing toon ledge is composed of "plan 
fragments" which ara abstract and rattier general plans for accomplishing 
goal* r An example from Sus-soan's blocks world program writing iysten is: 
*To achieve x and y, try to achieve them separately." Another simple 
example is the fragment: "To achieve x. if we have a KMthod of achieving JC 
given that He have y, first achieve y, tLne:i uie tin; method for x." Still 
another is: "To achieve 'There are ne *'s with property P, ' find each x 
with, property P and achlave the negation of P on k. h These rules are in fact 
only heuristic and Likely to result in bugs. Ihnugh the fragments shewn are 
abstract fragments are intended ta represent concrete knowledge as well. 
Programs are constructed by expanding and instantiating plan fragments. At 



• tcli level, which of the applicable plan fragnants is used to achieve a 
particular gcal is * design decision* A plan fcr a program is, at laast in 
part, the tree of expansions of plan fragments, with particular design 
daemons, used to create the pro gram. Since plan fragments often expand 
into bugs. *e also need notations showing what constraints were added (such 
at ordering) which were not parti of the original dasign, and what bugs they 
wars Supposed, to eliminate. 

Programs An Ons Example Of Tite lot ton Of 0e J tie rote Artifact 

It is probably clear that plans fer program* cm becoae very complicated and 

studying the eplstemGlofly of programs directly nay be too hard, Programs 
have conditional control structures -- their configurations (how they break 
up into parts and how these parts are related) nay depend strongly on their 
inputs. They havo shared structures such as sub rout lues and a modification 
to such a substructure potentially affects all of the places that 
substructure is used, Programs also deal with rather complex encodings Of 
In Formation -- there are many kinds of data structure * prngramwr nust know 
how to manipulate. 

Thesa facts are discouraging, hut it has become clear to us that the 
notions of plan fragoents and plans, and using such concepts in design, 
analysis, debugging, and explanation, are definitely not unique to the 
domain of programming but are common to reasoning about any "deliberate 
artifact," We have come to feel that we can make substantial progress in 
the Study of programs and their properties by first examining the nature; or 
thesa concepts in other, simpler domains. 



There fs A Contlnuua Of Doominx For Studying Thv EtnsteniDlo$it Of Deliberate 
Artt/scti 

Wa have searched for a Simple domain HhiCh eliminates many of the problems 
posed by the Study Of prog/rareming but which also deals with complex 
deliberate artifacts composed of tinnier artifacts deliberately arranged. 
In our search we wanted to avoid aeveral difficulties: the problems of 
spatial reasoning, visual recognition, and natural linguistics, all of which 
Involve other deep, deflecting issues. 

M NO see It, all of this argues Tar diving, into the following at a 
carefully selected point: 

■ The simple and ^nch sf.ic|;od monks worl3- 

■ One of the nany engineer: r.g arflas that can be approached 
through the relatively simple mathematics c-f luaoed 
paranater linear systems, The&ft include the design of 
electronic circuits,, many simple mechanical devices, and 
hyd ro 1 ic sys t eras . 

■ One of the nany engineering areas that makes use c-f 
Interconnections of subsystems characterized as ported, 
linear, black box information or materials processors. The 
lLinped parameter linear systems mentioned above ire to be 
understood to be a subset of this class for which the port 
descriptions are constrained to a certain simple fOTB, 
Almost all forms of engineering have a strong involvement 
with this kind of analysis in the course of design. 



■ One of tha areas In which the black boxes are non-linear. 

■ Pragmas , 

In tha mark done 40 Nr, tM CQiieens rations has been on the design, of 
lumped parameter linear systems for several reasons. It is well understood 
by engineers of all sorts and It his a precise technical vocabulary along; 
with interesting problems- The criteria for success are clear. The lumped 
parameter elements interact in stereotyped ways only at designated ports. 
Even meeehanical engineers r who are often stuclc with geometry 4 still make 
use of these principalis although they are limited in the complBStity they can 
successfully handle relative to, say, the electronics circuit designer - 

Hucli li Needed 

Our work to date, executed in the study world or electronics da si on, 
has confirned e need for the following obvious things: 

1 The need fur knowledge representations adequate to define 

system requirements. 

■ Tha need for transformations among representations, in this 
case magnitude graphs to pole-zero plots to algebraic systen 
function form, for example r 

I Tha need to match plan fragments in the bag of tricks 
against system requirements.. There is an implied need to 
understand how to match when the matches are not exact- 



■ Till need to debug the ufid&iirabie Interaction* that follow 
from first approximations which ignore fragment 
interactions. In this case, the Loading of one stage on 
another. 

All these <rr* neeeiJOfr. aw GffiiaLre, to expert problem JoitfiHfl tn general 
aince tflejf finite tfiKJiOflUea i jt ff«p rfciiraiJl Cfcal CGlittf fte cfasii/ied » o dcnurfji 
d/ sfeiiicrote artifact* - 

Figure 12 shows the kind of control structure we feel is required for 
the kinds of" problem we have in mind, Note that there is often a "BiB of 
Tricks 11 — a data base referred, t« by ail parts of the pro gran - 

Tftef* /J d Pint For Pntfe rat unfair £"el tififdte Afti/acta 

The Engineering Problem Solving Project is concerned in the short run with 
electronic design (either ic or discrate}- Later it is to blend into an 
attack on. the structure of mare compile* ted artifacts t such as programs. In 
this section, we outline our timetable. 
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ft Gerald &isJtfTian H Richard Stallman, and Gerald Eoylance have 
written and extended EL, a program far circuit analysis. 
Thi* program uses its Knowl&dfl* of the plan Structure of the 
analyzed device to avoid blind relaxation techniques in 
analyzing circuits. As of now, EL handles quiescent circuit 
an A lysis- Sussman et tfi ire currently engaged in extend ing 
£L to AC signal anulyils an- reorganizing It toward greater 
flexibility. By June l»76 they plan to have a finished 
product. EL's ability to "explain" its conclusions will be 
critical. The system will not only give answers on demand, 
it will explain these answers as well. 

■ Currently, flood progress is being made on electronics design 
debugging, Allen uriwn haa finished a Ph.D. dissertation 
on "Localisation of Fiailures in Radio Circuits." This work 
will be extended by Matt Hason, who will inpltment Brown's 
ideas. He Should have Browi H S LOCAL prcgraa srudud by June, 
1*76, 



I Drew HtDarmett Is studying the problem of interface design. 
He has written fl very simple prooraii that Unows about a fe*f 
Of the linear system plum fragments. TherB are two Blin 
subproblens that have arisen; choosing transforation 
techniques for problems; and satisfying conflicting 
constraints in putting plan fragments together. H* will 
complete a Ph.D. dissertation on these topics by June 1976- 
HcDermott is doing his research uno-Er a commitment to a 
certain Hind of modularity in programming . He would like 
his prograa to have "wired in 1 the theory of artifact 
design, hut the plan fratriuMits , problem analysis 
tachniqucs, and other "advice" are to be more loosely 
coupled to the prograa- 

■ There reoains the goal of understanding the design and 

modification of complex programs. In parallel with our 
electronics wort, Charles Rich and Howard Shrobe have been 
studying computer programs directly, in terms or engineered 
artifacts, like circuits. They ire aiming at a progran to 
aid a human in designing hash tables t by June 1976. 

■ The Kich and Shrobe work is part of a larger goal of the 
development of design analysis tools for program.* analogous 
to the analysis tools Used in lumped parameter and linear 
systems analysis. 

For reasons we have mentioned, progra^-iir,;; is a very difficult domain ^ 
It is hard now to predict what we will have learnftd from another engineering 
discipline. In e year, we will be in a good position to bring our 
intuitions and experiences tD bear on program design. 



Vm bmHtitt thdl tht mrk oytlined 1 in this JfCtiOfl is ON* ttmip o rt#Jt t of 
vhct is Jrecej^ffrir before at Cdfl (teifff" systems iJiirfc can r*d!Jtf lirttfl tfictr 
OHfK prografas- . The Other COnpQIKint needed, it seeas to U3> ti the thorough 
understanding oT Che (loan ins for which the automatic systems are to 
programmed. This is one raason why we applaud Martin's work, in Project HAC . 
Vfe believe our effort to be complementary to hts- 

PERSONNBL 

■ Work in basic vision studies is. under the general direction 
of DAVI& HARfc. His theories of vision are on the critical 
path to a batter understanding of vision. Professor 
BERTHOLD K. P. HORN, wllOSi first responsibility i* to direct 
the applications of Vision and Banipulation worft, will 
continue to contribute from tine tD time. 

■ SHIMON ULLHAN, who iust recently completed a sucessful 
study of light source detection, will continoa to pursue 
aspects of the Karr theory. Other students with likely 
involvement include KEN FOUR'S, an undergraduate Who has 
helped Harr with programming for soma tine and JAN 
aALK.DW&KLl , a graduate student wfio has Just completed a study 
of electronic component identification. 

l Professor PATRICK H. WINSTON plans to teite an active role in 
working with students en tin? comnon sense physical reasoning 
problem. 



HIKE FREILING hei proposed to wort on the coiuaon sense 
raasoning and analogy nfl^hanisns appropriate to basic 
nachlnery, RICKAM fiROUTj, hoping to deal directly with the 
problem Of analogical reasoning, wlLL try tD expose key 
issuer by Studying thft Kay solid geome'.ry prohlecs are 
solved, throgh plane ga one try analogies. JOKAN" DEIUEER, 
working under tha supervision of Professor SEVJlOUTt PAPERT, 
is working on the qualitative knowledge/quant i tative 
knowledge interface. BEN HJIPEU U doing mental maps. 

Prafesior GERALD' SUSSBAJI is in ch&rfle of making progress la 
the general domain, of expert problem solving, program 
understanding system, and tile api sterol cgy of "deliberate 
art if acts." Soma of this work is dona in cooperation with 
Professor CARL HEWITT, supported mostly through Project ftAC 
and partly through the Artificial Intelligence Laboratory. 

Sussnan and Hawitt are jointly supervising CHARLES RICH and 
HOWIE ECHROBE, who are working on the problen of expert 
prohlen solving using an expert in pregramning a* the 
discovery domain, 

Sussman has worked closely with RICHARD SIALLftAN and GERALD 
ROYLAfJCE on the development of the electronics expert, EL. 
Roylince plans considerable further work in this dir action, 
expecting to- introduce snail Signal analysis to COQplanent 
the quiescent analysis expert iaplemented so far.. 
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SYSTEMS AND SYSTEMS CONCEPTS 

The Artificial Intelligence Lahoretory wishes to develop a low cust system 
intended to serve single users with en outstanding: LISP together with 
equally high quality text editing, debugging, and display features, The 
overall goal is to produce a computing sy&tea dramatically superior to any 
available today bath in terms af p-Ower ani CCtt, We intend to achieve this 
goal by way nf implementing a prototype system manifesting ideas now worked 
out and detailed in appendices. 

We also inter.:! to do some thinking about vision processors with a view 
toward preparing Tor the time when such machines ire both necessary and 
possible. 

PURPOSE 

This section is short since other appanded documents fully convey tho 
general spirit and technical detail behind this effort. These few words 
will simply summarise the salient points. 

TflC LHP flfftftifle Prvmi&ts A A'em Way Of Tainting About CortrpUlfrtO 

The trend has been toward ..cheap hardware and expensive software, in 
contrast to the classical shared CfrU and central oeaory systens, the LISP 
□achine Concept exploits the Cheap hardware trend. A principal result of 
the overal l approach ii simpler system software, The impact on huw 
copputing is done will he, we believe, at least es dramatic as the 
introduction of tine-sharing. It is perhaps not surprising that such a 
development should spring up from the connjnity of people who think about 
Artificial Intelligence itiasituch S3 Artificial Intelligence development has 
always made extreme, technology-pull demands on computer system science . It 



Is argued, after all h that tine-sharing itself was largely incubated and 
promoted by early work in artificial intelligence. 

Tht LISP /facMne 0//erj Wore Power 

The family of idea* that make the HBP machine an Rxc-.ting project 
anticipate the future, Paramount among these ideas may fie the large address 
space offered. We believe that 1D24K af address Space will be needed 
eventually in situation* wftqre natural English input Is demanded. We base 
this on, estimates that seea, sensible in tenas of Winograd's natural language 
wort In the simple blocks world which required 150K and on subsequent work 
by Nartin which. Leads him to talk in tarns of 51ZK as a minimum Existing 
and proposed PDP-1Q type Bachine* allow only IB ef the ZO bits of address 
needed, while the LIS? machine is to provide Z3, a comfortable margin. (The 
KL-10 processor will address more than 19 bits af address space, but only by 
way of an awkward afterthought method forced by design decisions which were 
reasonable at the time the PDP-10 architecture was introduced year* ago.) 

We believe that natural language is not alone in suggesting the need. 
5o far programs have worked with sophisticated programs on small data bases-. 
As wa become nore ambitious and mare concerned with real problems, the 
larger address spaces will be not just desired, but essential. We note* ft& 
an example of this, that the CON'HIVtR interpreter alone occupied over 100K 
of central nemory and therefor* points toward systems containing it which 
■ust be beyond the reach of the ZS6K, IB hit address space, 

LISP Kachinc 0e ire E jme N t Hilt £e Surprisingly Intxptltiivc 

Ten years ago it would have been foolhardy for a university group- to propose- 
the creation of a grand new computing engine. Titles have changed. Large 
scale integration has given us prefabricated walls where before there was 



only brick and mortar- Five years ago at least ten tines as many chips 

would have been needed. Moreover, Stanford and fl.l.T. work on interactive, 
graphically oriented design aids has jIveii us autoaat ically generated 
documentation and Wire wmjjplnu proiToiis where before we had only drafting 
mistakes, lost bits of paper p md Chaos, Figure 13 Illustrates the sort r< f 
documentary output automatically generated in the course of the design' 
process. These new tools have been thoroughly demonstrated through use on 
projects of demanding scape, including the design of the new PEC KL-1D 
processor. 

TAb LISP Xochinc ViU Serpe T^e feeds OJ florrjf 

Thu time has cowe to produce such a machine because the technology Is reedy 
and the users breathless- Certainly the artificial Intelligence R.&D 
community is the first source of customers. For this group, not only are 
the technical advantages extrene, hut alsg the new ability to get started 
with an Initial Investment one-tenth the size of a PDF-- ID configuration is 
extremely attractive. Smooth expansion via one-by-one, addition of LISP 
machines substitutes for big ticket decisions. Mora people, widely 
SUpporte-d, will do Artificial Intelligence work. Of course the scope or the 
predicted imp-act reaches far beyond Artificial Intelligence for two reasons; 
one is that the predicted infiltration of Artificial Intelligence into most 
Of the frontier problem arEas of computer science has occurred!, another is 
the feet that the LISP machine itself, whll& worked out with LISP in mind, 
will certainly support other languages with largely the same advantages, 
The concept will make life easier and more cost effective not only for the 
Artificial Intelligence LISP using comwunity, but also for thos-e who for 
historical or genuine technical reasons oust stick with the more classical 
languages for production and execution of end user software. 

It particular, we beiitve MSP mtfcMfl&j, being vent cheap in JG jreeTJ 
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ttJiie, wlU be tAe hardware. Joandutian for tH eflffliJUter ujer'a IMTiiUGENT 
T&HHINAL, ifl* manager •* HASAGEttENT ASSISTANT, tf\e * indent or trainee's 
PERSONAL PEPAQOGUE. the /iretorp'j INTELLIGENT ASSEMBLER, and, in cccii, 
everyone's PERSONAL HELPER. 

THE LISP MACHINE CONCEPT 

Thi hardware configuration is diagranraed in figure 14, It li to consist of 
the fallowing core; 

■ A PDP-11 for doing I/O, for controlling a unibus, and for 
aiding in debugging other companding principally the CCNS 
processor r 

■ Tha CONS processor for executing LIS? tailored instructions. 
The COKS processor successfully integrates many pood ideas, 
Dmonu thea protframciafcle. microcode, LISP oriented b-yts 
handling operations, Special list processing dire clad 
addressing nodes, a fast internal stack, and more. The COh'S 
machine is fully described In an appendix, 

■ A good deal of central memory, probably on the order of 50 K 
to 104K 16 bit word*, This large amount of non-shared, 
dedicated local menory is deternined by the overall design 

philosophy which rejects classical time sharing in favor of 
system simplicity and performance. 
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■ A fast local disk or possibly CCD bulk senary of the sort 
recently heralded by the Introduction, of 16K. CCD chips. 
This will hold that part of the Z3 hit address space not U 
■fast" meetory and frill bo the primary source of system 
loft>ijrB And user prour^mi. 

■ A connection to the re.it of the computing world seen by the 
machine. Here li where the sharing horizon la to stand. 
Some Unit, probably seme sort gf Eth-er net, connects users 
together, provides for data and program sharing, and ill the 
usual cooaunity features introduced by tine sharing. 

COMPUTING ALTERNATIVES 

Several other competing recommendations for. seating computation needs have 
been considered. There are good arguments for some although we believe the 
weight of the evidence favors the LIS? machine. Here are the principal 
alternatives. 



■ Option 1: SlnpLy wait for the ISG.vufl POP-ID, The extensive 
software work already done makes this A reasonable position. 
Unfortunately the one* state-of-the-art P&P-lfl arehltaetura 
now croaks with aga. Tht snail address space is 
particularly worrlsoae. V* do see, however, an intermediate 
time during which LISP pachipes will stand parasitic oh PDP- 
11} syateoa wilting for users ready to run LISP. Itie PDr-tO 
will continue to run the excellent text editing end general 
support loftware already Misting, while LISP execution 
takes advantage of the LISP machine's dramatically batter 
ideas for running .LISP- This is the reverse Df the usual 
idea which fares out the secretarial wari ta PDP-lls, 
presumably running yet unwritten secretarial software. 

■ Option 2: Develop a PDP-11 LISP, We have done this, but 

intend no najor worX dependent on It, The address space 
available to the PDP-11 architecture forbids It, Such a 
LISP can run some snail demonstration end application 
program, but only a small portion of those being written 
now by workers In Artificial Intelligence. 

■ Option 3: Finish the LISP machine, This saans the wise 
Course. 

LISP MACHINE OBJECTIVES 

Our Intention 11 that the LISP nachin* development be canpleted during the 
next two year* with a prototype running, by January,. I9?e and an advanced, 
prototype running oy January, 1977- 



LISP MACHINE HISTORY 

Richard Greanblett conceived the idea, of the LIS? machine. Hi was scan 
work i no in dose cooperation with Tnm Knight. 

The work proposed here was earlier proposed as a Joint project between 
the Artificial Intelligence Laboratory and Project MAC, Joel Moses of 
Project MAC wrote the original proposal- As now organized, however, the 
work will be handled within the Artificial Intelligence Laboratory with a 
view toward clou supporting counsel and communication with Project HAC, 
which is to be regarded as equally responsible for getting the ideas off the 
ground. It seems clear that MACSYP1A and other Project MAC interests will be 
best served in the future by the liSP Mac'-ii-i* or seething very similar. 

PERSONNEL 

Thii project is in the hinds of people well known ror their ideas and 
preeminence In system software and hardware design, RICHARD GREEK BLATT, 
principal author of the ITS time-sharing system, is in charge of the 
software, THOMAS KNIGHT, principal author of the Knight terminal display 
sy&tem, Innumerable interfaces and special purpose processors , end overall 
coordinator of our digital hardware syttflc.5., is in. charge of the hardware ^ 
They will work with JACfc HOLLOHA^, well known for his wort on the so-called 
Superfoonly Project at Stanford which was extremely influential in the 
development of the latest in the P&P-lQ family of conputers, the KL~L0 P and 
with JOHN H0U550URIS, who IS now finishing up a parallel processor for chess 
(not funded by ARPA). BILL FREEMAN will interface the LISP Hachine to the 
existing tine sharing system,. 



A VISION MACHINE 

Soon we will know enough and prices of processors will be sufficiently low 
for sensible proposals to be made for very general parallel image processing 
con ri.gur&t ions. He are therefore giving -some thought to what could be done, 
particularly with the recently announced LSI- Us. This thinking is not Tar 
along and it is inappropriate to make a sharp proposal at this time, We do 
append a working paper on the subject, however, to show that such a project 
deserves some thought, 

APPENDICES 

HORN AND UlttSTOK: PERSONAL COrlPUfLRS (REPFUN7ED FRDJl DATAMATION rtA&AZINE) 

HOSES r ORIGINAL LISP HACKING PROPOSAL 



KNIGHT; THE CONS HlCftO PROCESSOR 
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